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SUMMARY  OF  FINDINGS 


A.  ,  INTRODUCTION 
- ^ — 

The  Trl-Servlce  Medical  Information  Systems  (TRIMIS)  Program 
Office  (TPO)  has  installed  four  initial  stand-alone  computer  systems 
for  support  of  radiology  operations  in  Medical  Treatment  Facilities 
(MTFs) .  The  system,  known  as  the  Tri-Service  Radiology  System  (or 
TRIRAD) ,  provides  automated  support  to  patient  management,  scheduling, 
film  management,  administrative  reporting  end  statistics,  teaching  and 
research,  and  reporting  assistance  to  radiologists.  The  experience 
with  the  Implementation  of  this  system  in  the  Radiology  Department  at 
the  Naval  Rospital,  Bethesda  (NHB) ,  has  been  evaluated  in  order  to 
provide  information  for  use  in  decision-making  about  the  future  use  of 
automation  In  radiology  departments  in  other  MTFs^ 

The  evaluation  was  based  upon  a  comparison  df  processes  and  data 
on  operations  collected  in  November  of  1981  before  the  system  was 
Installed  at  NHB  (baseline)  with  similar  information  collected  mfter 
the  system  was  In  routine  use  (post-implementation)  in  November  of 
1982.  This  information  was  supplemented  by  information  gained  during 
five  visits  to  the  site  to  monitor  the  status  of  system 
implementation. 

During  the  baseline  and  post-implementation  data  collection 
periods,  quantitative  studies  were  conducted  concerning  the  most 
likely  major  impacts  of  TRIRAD  on  the  operation  of  the  Radiology 
Department :  „  * 


a  turnaround  time  for  radiology  reports, 

e  labor  and  process  times  for  segments  of  the  process  of 
providing  radiology  services ,  and 
e  film  file  availability. 


More  intensive  studies  were  performed  in  Main  Radiology  and  the 
Imaging  sections,  which  accounted  for  the  major  share  (7&Z  in  FY81, 
and  832  in  FY82)  of  the  Department  workload.  Other  impacts  were 
studied  by  observation  and  by  use  of  surveys  and  interviews,  including 
satisfaction  of  Radiology  Department  staff  and  attending  physicians 
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with  radiology  services,  perceptions  of  staff  concerning  changes 
resulting  from  TRIRAD,  and  staff  acceptance  of  the  system. 
Information  on  costs  was  also  obtained  from  Department  budgets  and 
TRIRAD  contract  documents. 

The  Information  gained  has  been  analyzed  and  is  summarized  below 
as  it  relates  to  the  goals  and  objectives  established  by  the  Medical 
Review  Group  for  TRIRAD,  the  experience  with  computer  assistance  to 
reading  and  reporting  functions,  and  staff  acceptance  of  the  system. 

B.  SYSTEM  GOALS  AND  OBJECTIVES 

1.  Goal  1;  To  Provide  Accurate,  Timely  Radiology  Reports 

Quantitative  studies,  survey  results,  and  interviews  all 
Indicated  that  there  were  substantial  improvements  in  the  timelinees 
of  radiology  reports  from  the  baseline  to  the  post-implementation 
period. 

During  the  implementation  period,  the  backlog  of  examinations  for 
which  no  report  had  been  distributed  (incomplete  reports)  decreased 
steadily,  as  the  timeliness  of  completion  of  reports  improved.  In 
Main  Radiology,  for  example,  the  turnaround  time  from  patient  arrival 
in  the  Department  until  a  typed  or  printed  report  was  available  for 
distribution  was  reduced  from  16  days  in  the  baseline  to  6  days.  The 
proportion  of  the  volume  that  was  handwritten  final  reports  in  Main 
Radiology  also  dropped  from  95%  to  6%.  In  Orthopedic  Radiology 
handwritten  reports  were  prepared  for  all  interpretations. 

Bar-coded-reporting  assistance  was  used  for  20%  of  all  reports 
prepared  in  the  Department,  and  appears  to  have  contributed  to  the 
decrease  in  turnaround  time;  17%  of  these  required  no  transcription 
while  3%  of  the  reports  were  prepared  from  a  combination  of  bar-coding 
and  dictation.  In  other  sections  of  the  Department,  all  reports  were 
transcribed  from  dictation  in  the  baseline  and  printed  by  TRIRAD  after 
system  Implementation.  Associated  baseline  mean  turnaround  times  in 
the  range  of  15-43  days  had  been  reduced  to  5-14  days  by  the  time 
post-implementation  data  were  collected. 

Satisfaction  of  both  Radiology  Department  staff  and  attending 
physicians  with  the  timeliness  of  interpretation  reports  increased. 
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Both  groups  reported  that  access  to  information  concerning  patient 
examination  status  and  reports  stored  in  the  TRIRAD  system  was  one  of 
the  most  significant  benefits.  Attending  physicians  noted  that  they 
were  more  likely  to  find  radiology  reports  in  patient  records  than 
before  TRIRAD  was  implemented;  however,  they  also  reported  that  TRIRAD 
had  not  eliminated  all  problems  with  report  availability.  (As  TRIRAD 
p  is  a  stand-alone  system  and  Installed  only  within  the  Radiology 

Department,  its  influence  on  certain  segments  of  the  process  and  on 

*  distribution  and  filing  of  reports  in  patient  records  outside  of  the 

p  Department  may  only  be  indirect.) 

The  reasons  for  the  reduction  experienced  in  report  turnaround 
time  are  believed  due  primarily  to  a  commitment  on  the  part  of 
Radiology  management  to  make  some  changes  in  operations  and  staffing 
in  order  to  improve  turnaround  time  rather  than  due  solely  to  TRIRAD. 
The  most  important  change  was  in  bringing  transcription  services 
within  the  Department  by  adding  two  transcriptionists  to  the  Radiology 
Department  staff  rather  than  using  outside  services. 

Both  Radiology  Department  staff  and  attending  physicians  were 
satisfied  with  the  accuracy  of  the  interpretations  in  radiology 
reports  in  the  baseline  period.  This  level  of  satisfaction  did  not 
change  during  the  post-implementation  study.  A  few  staff  felt  that 
TRIRAD  had  affected  accuracy,  but  this  appears  to  be  related  to  the 
Increased  accuracy  of  information  contained  on  the  printed  report, 
e.g.,  patient  identification  or  examination  date,  and  not  due  to  the 
accuracy  of  the  interpretation  per  se. 

2.  Goal  2:  To  Collect  Management  and  Workload  Data  for  Use  in 

Optimizing  Radiology  Resources 

At  the  time  of  the  post-implementation  study,  the  Administrative 
Reporting  and  Statistics  module  of  TRIRAD  was  in  the  process  of  being 

*  Installed.  Therefore,  the  experience  with  management  information  was 
extremely  limited.  When  this  module  is  Implemented,  it  is  expected  to 
reduce  the  effort  required  to  prepare  required  management  reports  and 

*  to  make  available  management  information  that  was  infeasible  to 
compile  by  manual  methods. 
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3.  Goal  3:  To  Improve  Availability  of  Patlent-Specif ic  Historical 

Radiological  Reporta  and  Films 

In  the  baseline  process,  reports  and  flints  were  filed  together  in 
the  film  file  libraries.  During  the  post-implementation  period 
reports  were  stored  in  the  TRIRAD  system,  whereas,  only  the  record- 
keeping  concerning  the  location  of  film  files  was  automated  and  the 
hardcopy  film  storage  and  film-library  procedures  remained  essentially 
unchanged.  The  benefits  associated  with  changes  in  availability  of 
reports  were  thus  quite  different  than  for  films. 

The  staff  surveys  and  quantitative  studies  indicate  that  the 
result  of  these  changes  was  greatly  improved  availability  of 
historical  reports.  The  reason  is  that  once  a  report  was  entered 
into  the  system,  it  was  retrievable  at  any  time  on  demand.  Thus 
reports  could  be  accessed  more  easily  and  in  a  more  timely  manner. 
Both  Radiology  staff  and  attending  physicians  reported  Increased 
access  to  reports  to  be  a  major  benefit  of  TRIRAD. 

Availability  of  historical  film  files  showed  minimal  or  no 
improvement.  Though  staff  reported  that  the  ease  of  obtaining  film 
files  had  improved,  the  availability  of  a  film  file  still  depended 
upon  manual  filing  and  retrieval  in  the  film  library.  As  the  patient 
data  base  on  reports,  film  file  records  and  examination  history 
continues  to  grow,  availability  of  this  information  when  needed  should 
continue  to  improve. 

A.  Goal  A:  To  Make  Maximum  Use  of  Resources,  Equipment,  and  Staff 

Through  Efficient  Scheduling 

At  the  time  of  the  post-implementation  study,  the  automated 
scheduling  function  of  TRIRAD  was  not  fully  operational;  the  optimal 
scheduling  module  did  not  function  due  to  software  errors.  Never¬ 
theless,  the  remainder  of  the  scheduling  module  was  in  use  in  all 
sections  of  the  Radiology  Department.  Sixty  percent  of  Department 


staff  who  responded  to  the  post-implementation  survey  felt  that  TRIRAD 
had  improved  scheduling,  and  only  11%  believed  that  there  had  been  no 
improvements  (the  remainder  were  neutral) . 
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Staff  time  devoted  to  scheduling/reception  functions  increased 
somewhat.  However,  during  Che  scheduling  and  registration  processes 
in  the  post-implementation  period,  the  patient  data  base  was  created 
or  updated.  The  increases  in  staff  time  were  thus  believed  to  have 
been  due  to  entering  the  additional  data  required  to  meet  the 
increased  information  needs  for  the  patient  examination  history, 
rather  Chan  due  to  a  decrease  in  the  efficiency  of  scheduling. 

The  post-implementation  staff  survey  Included  questions  concern¬ 
ing  the  effect  of  TRIRAD  on  the  efficiency  and  economy  of  the  Depart¬ 
ment.  A  large  percentage  of  the  respondents  believed  that  TRIRAD  had 
made  their  work  more  efficient  and  chat  the  system  saved  them  time. 
However,  respondents  were  also  largely  in  agreement  that  neither 
personnel  requirements  in  the  Department  nor  the  costs  of  providing 
radiology  services  had  been  reduced.  This  perception  was  consistent 
with  the  results  of  an  analysis  of  Department  budgets  and  a  marginal 
cost  analysis,  which  demonstrated  some  shifts  in  resources,  but  no 
actual  resource  reductions  during  the  first  year  of  TRIRAD  operation. 

The  combined  experience  with  respect  to  scheduling  and  resource 
utilization  indicates  that  scheduling  was  Improved  and  that  some 
efficiencies  have  been  gained  in  work  performed  in  the  Department. 
However,  the  anticipated  relationship  between  these  two  changes,  as 
specified  in  the  Medical  Review  Group  goals,  has  not  been 
demonstrated . 

5.  Goal  5;  To  Reduce  Patient  Waiting  and  Processing  Time  due  to 

Better  Scheduling 

Quantitative  studies  included  measurement  of  the  elapsed  time 
between  patient  arrival  in  the  Department  for  examination  and  patient 
departure.  The  data  indicate  that  this  time  increased  somewhat  in 
five  sections  and  decreased  in  three  others.  However,  the  increases 
are  believed  to  have  resulted  from  changes  in  procedures  for 
requisitioning  files,  changes  in  the  mix  of  examinations  and  chanaes 
in  staffing  rather  than  from  improved  scheduling  or  other  direct 
effect  of  TRIRAD.  Thus,  the  experience  with  patient  waiting  and 
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examination  time  was  mixed  and  appears  to  have  been  determined  for  the 
most  part  by  factors  unrelated  to  TRIRAD  scheduling  functions. 

6.  Goal  6:  To  Reduce  the  Number  of  Repeat  Exams  as  a  Result  of 

Improved  Film  File  Accessibility 

Film  file  accessibility  was  evaluated  both  by  quantitative 
studies  and  surveys  and  interviews.  Survey  results  suggest  some 
Improvements  in  accessibility  of  current  films  (recent  films  for 
outpatients  or  films  for  inpatients)  as  a  result  of  the  film-file- 
tracking  functions  of  TRIRAD.  Staff  in  the  Department  and  attending 
physicians  also  reported  that  the  ease  of  obtaining  films  had 
Increased.  However,  quantitative  studies  of  film  file  availability  in 
reading  rooms  and  surveys  and  interviews  consistently  indicated 
Esinimal  or  no  improvement  in  the  accessibility  of  historical  films. 
As  the  film-file-data  base  continues  to  grow,  accessibility  to  these 
films  may  improve.  The  lack  of  a  discernible  improvement  is 
attributable  to  the  fact  that,  though  TRIRAD  automated  the  bookkeeping 
functions  of  film-file  management,  the  films  themselves  were  still 
filed  in  "hard  copy."  Until  new  technology  is  available  to  store 
Images  digitally  on  the  system,  problems  with  the  availability  of 
films  will  not  be  totally  solved  by  any  radiology  management  system. 

The  issue  of  repeat  examinations  was  included  in  the  post-imple¬ 
mentation  survey  of  Radiology  department  staff.  Thirty-eight  percent 
of  respondents  reported  that  fewer  repeat  examinations  were  required 
(38%  were  neutral,  24%  felt  no  impact  had  occurred).  Given  the  lack 
of  discernible  change  in  accessibility  of  historical  film  files, 
reduced  turnaround  time  for  reports  (discussed  under  Goal  1)  and/or 
increased  access  to  patient  information  (discussed  under  Goal  3)  are 
more  plausible  as  the  causes  of  the  reduced  need  to  repeat  examina¬ 
tions  reported  by  some  department  staff  than  film-file  accessibility. 

7.  Goal  7;  To  Reduce  Personnel  Time  Spent  in  Clerical  Transcription 

One  of  the  most  significant  impacts  that  resulted  from  the 

Introduction  of  TRIRAD  and  accompanying  management  changes  was  the 
Increase  in  printed  (rather  than  handwritten)  interpretation  reports 
from  5%  in  the  baseline  period  to  94%.  Some  of  these,  17%  of  the 
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Department  volume,  were  prepared  directly  by  the  radiologist  using  a 
bar-code  reader  and  averted  the  need  for  transcription.  The  overall 
Increase  in  transcribed  reports  amounts  to  nearly  30%  of  the 
examination  volume  (35%  were  transcribed  during  the  baseline  and  65% 
were  transcribed  during  post-implementation).  On  this  basis,  time 
devoted  to  transcription  Increased. 

However,  the  efficiency  of  transcription  clearly  Improved. 
Following  the  introduction  of  TRIRAD,  two  typists  were  added  to  the 
Department  staff  and  all  transcriptions  were  performed  within  the 
Department  rather  chan  by  the  hospital's  Central  Transcription  service 
and  an  outside  typing  service.  (Department  staff  felt  that  typing 
efficiency  was  probably  increased  due  to  the  increased  use  of 
pre-defined  text.  While  pre-defined  text  was  available  prior  to 
TRIRAD,  its  use  appeared  quite  limited  during  the  baseline  study.) 
The  elapsed  time  for  the  transcription  segment  of  the  process  was 
reduced  from  7  to  21  days  to  less  than  1  day,  a  reduction  that 


contributed  significantly  to  the  reduction  noted  in  turnaround  time 
for  reports. 

C.  READING  ASSISTANCE 

Before  TRIRAD  was  introduced  in  the  Radiology  Department  at  NHB, 
radiology  reports  were  either  handwritten  by  the  radiologist  on  the 
request  form  or  the  interpretations  were  dictated  and  typed  by  outside 
transcription  services.  TRIRAD  gave  the  staff  radiologist  and 
radiology  residents  the  ability  to  produce  printed  interpretation 
reports  directly  from  input  via  a  bar-coding  system  or  an  optical  mark 
reader  (OMR) . 

By  the  end  of  the  first  year  of  system  use,  the  bar-coding  system 
was  being  used  by  a  small  number  of  radiologists  in  two  sections  of 
the  Department.  Although  measured  radiologist-reading  and  report- 

preparation  times  when  using  this  method  appeared  to  be  somewhat  ] 

shorter,  bar-coding  appeared  to  be  used  selectively  for  certain  simple  3 

interpretations.  Reductions  in  radiologist's  labor  for  reporting  j 

appear  not  to  have  occurred.  Computer-assisted  reporting  appears  to  j 

have  reduced  turnaround  time  and  eliminated  the  need  for  j 


transcription.  This  results  in  less  labor  devoted  to  transcription 
(an  average  of  2  minutes  per  report  in  the  post-implementation  period) 
and  decreased  turnaround  time  for  results  since  the  interpretation 
report  can  be  verified  immediately  and  printed  for  distribution. 

In  the  first  few  months  after  TRIRAD  was  installed,  the 
radiologists  made  greater  use  of  bar-coding.  The  motivation  appears 
to  have  been  to  reduce  the  backlog  of  reports  in  transcription  and 
produce  reports  in  a  more  timely  manner.  As  these  aspects  of  the 
process  Improved,  use  of  bar-coding  was  observed  to  decline. 

D.  .  SYSTEM  ACCEPTANCE  BY  RADIOLOGY  STAFF 

Fifty-nine  percent  of  respondents  to  the  post-implementation 
survey  of  Radiology  Department  staff  indicated  that  overall  thay  were 
satisfied  with  TRIRAD,  and  only  6Z  considered  themselves  dissatisfied. 

Many  Department  staff  felt  that  there  had  been  benefits  to 
patient  care.  In  particular,  they  cited  the  improved  access  to 
patient  information  as  a  significant  contributor  to  their  ability  to 
provide  information  on  examination  status,  reports  and  current  films. 
As  a  consequence,  they  believed  that  the  Department  assisted  attending 
physicians  in  providing  better  patient  care  and  the  attending 
physicians  were  more  satisfied  with  radiology  services.  Most  (79 X)  of 
respondents  to  the  survey  felt  that  TRIRAD  was  easy  to  learn,  and  many 
(59X)  believed  that  overall  the  benefits  outweighed  the  disadvantages. 
Seventy-five  percent  of  the  survey  respondents  further  believed  that 
the  benefits  from  TRIRAD  would  continue  to  increase  in  the  future. 
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I.  INTRODUCTION 


A.  OVERVIEW 

The  Tri-Service  Medical  Information  Systems  (TRIMIS)  Program 
Office  (TPO)  has  installed  four  initial  stand-alone  computer  systems 
for  support  of  radiology  operations  (Tri-Service  Radiology  System  or 
TRIRAD)  in  Medical  Treatment  Facilities  (MTFs).  The  experience  with 
TRIRAD  at  the  third  installation  site— the  Naval  Hospital »  Bethesda 
(NHB)*  in  Bethesda,  MD— has  been  evaluated  in  order  to  provide 
information  for  use  in  decision-making  about  the  future  use  of 
automation  in  radiology  departments  in  other  military  health  care 
facilities  and  to  identify  system  and  operational  changes  that  might 
lead  to  realization  of  additional  benefits  at  NHB  or  at  other  sites. 

This  evaluation  has  been  conducted  in  several  phases.  Intensive 
data  were  collected  concerning  aspects  of  radiology  operations  at  NHB 
before  the  system  was  installed  (baseline  or  X  Period  in  November  of 
1981) .  The  implementation  of  the  TRIRAD  system  was  monitored 
periodically  from  November  of  1981  to  October  of  1982.  The  third 
phase  was  a  second  period  of  intensive  data  collection  concerning  the 
computer  (post-implementation  or  Y  Period  in  October  of  1982)  mode  of 
operation  once  the  TRIRAD  system  had  been  implemented  and  was  in 
routine  use.  This  report  presents  the  results  of  all  of  these 
evaluation  activities. 

B.  THE  SETTING 

The  Naval  Hospital,  Bethesda  is  a  very  large  and  modern  medical 
center  located  in  Bethesda,  MD,  near  the  Uniformed  Services  University 
of  the  Health  Sciences,  with  which  it  is  closely  allied.  As  a  major 
teaching  hospital,  it  provides  post-graduate  training  in  all  clinical 


As  of  September  1982,  the  name  was  changed  from  the  National  Naval 
Medical  Center  (NNMC)  to  Naval  Hospital,  Bethesda.  The  latter  is  used 
throughout  this  report,  though  the  facility  was  known  as  NNMC  when  the 
evaluation  was  begun. 
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areas.  A  variety  of  patient  care  and  research  programs  are  conducted 
in  collaboration  vith  Walter  Reed  Army  Medical  Center  and  the  National 
Institutes  of  Health. 

NHB  serves  a  local  catchment  population  in  the  Washington,  D.C. 
region  and  also  serves  as  a  regional  tertiary  referral  center.  The 
normal  authorized  operating  capacity  of  474  inpatient  beds  dropped 
briefly  during  the  evaluation  period  (summer  of  1982)  to  211  beds,  but 
had  been  restored  to  a  more  nearly  normal  level  of  420  beds  by  the 
time  post-implementation  data  were  collected.  The  outpatient  volume 
in  the  numerous  clinics  at  the  facility  averages  more  than  1500  per 
day. 

The  Radiology  Department  supporting  this  patient  load  is  sizable. 
Occupying  an  area  of  about  13,600  square  feet,  the  various  sections  of 
the  Department  are  dispersed  throughout  several  floors  of  the 
facility.  The  full  range  of  specialized  radiology  services  offered  is 
reflected  in  the '  nine  individual  sections  operating  within  the 
Department:  Main  Radiology,  Gastro-Intestinal  (GI)  Radiology, 
Orthopedic  Radiology,  Urologic  Radiology,  Special  Procedures,  Nuclear 
Medicine,  Computerized  Tomography,  Ultrasound,  and  Radiation  Oncology. 
Staffed  by  24  radiologists  and  residents,  70  technicians,  3  physicists 

and  15  clerical  and  support  staff,  the  nine  sections  performed  100,300 

2 

examinations /treatments  (equivalent  to  532,000  weighted  work  units*") 
in  FY  1981  and  102,000  examinations/treatments  (852,770  weighted  work 
units2 3)  in  FY  1982. 

C.  INTRODUCTION  TO  RADIOLOGY  SERVICES 

For  the  reader  not  familiar  with  the  typical  workflow  involved  in 
providing  radiology  services  in  a  military  MTF  with  complete  radiology 
services,  the  following  is  a  brief  overview  of  a  typical  manual 

2 

For  purposes  of  manpower  planning,  examinations  are  assigned  a  weight 
on  the  basis  of  complexity  (e.g.  a  standard  chest  X-ray  ■  3  and  an 
angiogram  -  28) . 

3 

In  FY82,  the  Nuclear  Medicine  Section  of  Radiology  at  NHB  adopted  a 
major  change  in  procedures  for  computing  weighted  work  units;  a 
nuclear  medicine  examination  averaged  10  work  units  per  examination  in 
FY81  and  57  work  units  per  examination  in  FY82. 
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process  and  Che  changes  commonly  made  to  that  process  when  Che 
Radiology  Department  is  automated. 

1.  Manual 


The  process  of  a  radiology  department  begins  with  the  arrival  of 
a  patient  for  examination.  In  some  departments ,  some  or  all  patient 
examinations  are  scheduled  in  advance;  typically,  only  the  more 
complex  procedures  are  scheduled,  with  the  routine  diagnostic  X-ray 
examinations  performed  on  a  walk-in  basis.  Patient  check-in  usually 
involves  checking  the  written  request  for  X-ray  examinations  (often  a 
multi-part  form)  and  in  some  cases  also  entering  pertinent  patient 
information  into  a  log. 

For  patients  who  have  been  examined  previously  in  the  department, 
the  film  file  is  pulled;  often  a  separate  file  is  maintained 
documenting  each  patient's  examination  history.  If  no  file  is 
located,  a  new  one  is  prepared.  Check-in  is  completed  with  the 
preparation  of  a  flashcard,  which  contains  pertinent  patient  informa¬ 
tion  and  is  used  to  identify  the  X-ray  film. 

After  the  X-ray  examination  has  been  performed,  the  film  is 
processed,  matched  with  the  X-ray  request  form  and  film  file  folder, 
and  given  to  a  radiologist  for  reading.  Departments  usually  have  a 
process  for  expediting  STAT  (emergency)  and  ASAP  (urgent)  examinations 
and  for  providing  a  wet-reading  (urgent  interpretation  and  report). 
The  radiologist  may  compare  the  present  film(s)  with  any  available 
prior  films  from  the  film  file  folder,  as  appropriate,  before  arriving 
at  an  interpretation  and  generally  prepares  the  written  report  in  one 
of  two  ways: 

e  by  handwritten  interpretation  on  the  request  form;  or 

e  by  dictation  for  transcription. 

Transcribed  reports  are  generally  returned  to  the  radiologist  for 
verification  and  signature.  The  process  is  completed  when  the  various 
copies  of  the  request  form,  including  the  typed  or  handwritten 
radiologist's  reading,  are  separated,  the  file  copy  filed  in  the  film 
file  folder  with  the  film,  and  one  or  two  copies  are  distributed  to 
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the  patient's  record  and/or  the  provider  who  requested  the 
examination. 

2.  Automated  Process 

When  this  process  is  automated  with  a  state-of-the-art  system 
such  as  TRIRAD,  the  major  changes  occur  in  the  reception/accession 
phase,  in  report  preparation,  and  in  file  management. 

At  patient  check-in.  Instead  of  working  with  the  logs  and 
hard-copy  files  typically  found  in  a  manual  process,  the  receptionist 
consults  the  video  display  terminal  (VDT)  and  enters  patient  registra¬ 
tion  information  for  a  patient  being  examined  for  the  first  time  or 
updates  the  examination  history  information  for  a  returning  patient. 
For  previously  scheduled  patients,  the  accession  process  amounts  to 
verifying  the  patient's  arrival  and  entering  the  information  into  the 
system  at  the  terminal.  The  system  generates  flashcards  and/or  film- 
file  labels  as  needed. 

The  radiologist  reading  the  X-ray  film  has  Increased  options  for 
preparing  the  interpretation  report,  such  as  the  ability  to  select 
predefined  standard  report  statements  for  generation  of  a  printed 
report  without  need  of  transcription.  Word-processing  capabilities 
are  Included  in  order  to  facilitate  the  transcription  process,  with 
verification,  editing  and  approval  by  the  radiologist  performed  at  a 
terminal. 

Once  entered  into  the  system,  the  patient's  examination  history, 
including  interpreted  reports,  is  stored  automatically,  with  the 
information  available  at  any  time  via  the  terminal.  This  reduces  the 
need  for  paper  copies  of  X-ray  reports  in  the  film  file;  hard  copies 
of  the  report  are  distributed  for  the  requesting  provider  and  the 
patient's  medical  record. 

D.  INTRODUCTION’  TO  TRIRAD 

The  Tri-Services  Radiology  System  (known  as  TRIRAP) ,  was 
purchased  from  National  Computer  Systems  of  Minneapolis,  MS.  It 
provides  state-of-the-art  automated  capabilities  as  described  briefly 
below: 
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Patient  Management — a  patient  data  base  including  patient 
identification  and  demographic  data,  examination  history, 
and  diagnostic  reports;  label  generation;  and  film-file 
tracking. 

Scheduling-scheduling  of  single  and  multiple,  same-day 
examinations;  generation  of  film-file  pull  lists  and 
administrative  reports. 

Film  Management— film-file  location  tracking,  including 
film-folder  transfers,  loans,  returns,  and  overdue  loans; 
film  transfers  accomplished  via  terminal  or  bar-code  reader. 
Administrative  Reporting  and  Statistics — generation  of 
reports  concerning  workload,  including  cancellation  list, 
day  logs,  incomplete  (exam  status)  list,  schedule  and  film- 
file  pull  lists,  and  film  inventory  report. 

Special  Interest /Teaching  Files — the  capability  for  creating 
teaching  or  research  files  from  the  patient  data  base. 

Reporting  Assistance — direct  generation  of  interpretation 
reports  by  radiologists  by  use  of  an  Optical  Mark 
Recognition  (OMR)  scanner  or  Stored  Radiological  Text  (SRT) 
via  a  terminal  or  bar-code  reader;  text-processing  support 
to  transcription. 

These  six  modules  of  TRIRAD  were  implemented  in  a  phased 
fashion,  beginning  in  November  of  1981.  By  the  time  post-imple¬ 
mentation  data  were  collected  in  October  of  1982,  the  system  had  been 
Implemented  throughout  the  Radiology  Department,  with  the  following 
exceptions: 

e  the  management/statistics  module  had  only  been 

partially  Implemented; 

e  the  scheduling  module  had  been  partially  implemented 
and  the  optimal  scheduling  module  was  not  functioning; 
and 

e  the  implementation  in  Radiation  Therapy  had  been 
deferred  pending  the  development  of  computer  algorithms 


Co  support  Che  specific  scheduling  and  reporcing 
requiremencs  in  chis  section. 

E.  GOALS  AND  OBJECTIVES  FOR  TRIRAD 

In  1978,  Che  Medical  Review  Group  of  Che  TRIMIS  Program  Office 
escablished  seven  goals  and  objectives  for  TRIRAD  as  follows: ^ 

•  To  provide  accuraCe,  timely  radiology  reporcs 

e  To  collecC  management  and  workload  data  for  use  in 

opcimizing  radiology  resources 

e  To  improve  availabiliCy  of  pacienc  specific  hiscorical 

radiological  reporcs  and  films 

e  To  make  maximum  use  of  resources,  equipmenc  and  staff 

through  efficient  scheduling 

•  To  reduce  Che  number  of  repeat  exams  as  a  result  of 

improved  film-file  accessibility 

•  To  reduce  patient  waiting  and  processing  time  due  to 

better  scheduling 

e  To  reduce  personnel  time  spent  in  clerical 

transcriptions. 

The  Medical  Review  Group  stated  additional  goals  that  it  believed 
would  be  more  difficult  to  quantify,  including  improved  capability  for 
clinical  research  and  improved  availability  of  information  on  patient 
allergies. 

These  goals  and  objectives  determined  the  content  of  the 

evaluation  of  TRIRAD  at  NHB. 

F,  ORGANIZATION  OF  THE  REPORT 

The  body  of  this  evaluation  report  is  presented  in  four  chapters. 
Chapter  II  describes  the  methodology  used,  Chapter  III  documents  the 
Department  workload  during  the  study  period,  summarizes  the  baseline 
work  flow  in  the  Radiology  Department  and  describes  changes  that  were 
made  when  the  system  was  introduced.  In  Chapter  IV,  the  results  of 
data  collection  activities  are  described.  Chapter  V  discusses  the 
results  in  the  context  of  the  system  goals  and  objectives,  reading 
assistance  for  radiologists,  and  staff  acceptance.  Detailed  support¬ 
ing  data  appendices  are  included  at  the  end  of  this  volume. 


II.  METHODOLOGY 


A.  APPROACH 

The  basic  analytic  framework  for  the  evaluation  was  a  study  of 
key  aspects  of  the  operation  of  the  Radiology  Department  at  NHB  before 
and  after  the  TRIRAD  system  was  installed.  This  was  supplemented  by 
periodic  monitoring  of  system  Implementation  as  it  progressed  in  each 
of  the  various  sections  of  the  Department.  Implementation  monitoring 
served  the  dual  purpose  of  documenting  system  status  with  respect  to 
timing  the  post-implementation  study  and  providing  an  opportunity  to 
observe  other  changes,  unrelated  to  TRIRAD,  that  affect  radiology 
services  (in  effect,  the  confounding  variables  in  the  study). 

B.  EVALUATION  OBJECTIVES  AND  SELECTION  OF  AREAS  FOR  STUDY 

The  overall  objectives  of  the  evaluation  were  to  document  the 
experience  with  TRIRAD  at  NHB  and  to  ascertain  to  what  extent  the 
goals  and  objectives  for  the  system  were  met.  The  most  likely  major 
system  Impacts  were  selected  for  more  intensive,  quantitative  study: 

•  turnaround  time  for  radiology  services  (elapsed  time 
from  patient  arrival  for  examination  to  availability  of 
interpretation  report), 

•  elapsed  time  and  labor  devoted  to  segments  of  the  work 
flow  for  processing  patients,  examinations  and  reports, 
and 

•  film  file  availability. 

Other  impacts  were  evaluated  qualitatively  by  observation,  and  by  use 
of  interviews  and  staff  surveys  rather  than  by  original  data 
collection: 

•  staff  satisfaction  with  radiology  services, 

•  staff  perceptions  of  changes  resulting  from  TRIRAD,  and 

•  staff  acceptance  of  TRIRAD. 

Available  data  concerning  Department  workload  and  budget  were  also 
assembled  from  management  reports  and  TRIRAD  contract  documents. 

Turnaround  time  for  the  written  report  was  viewed  as  the  most 
significant  potential  impact  of  TRIRAD  on  radiology  services.  In 
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order  to  provide  a  basis  for  assessing  the  significance  of  changes  in 
turnaround  tine,  evaluation  staff  discussed  with  staff  radiologists  at 
NHB  what  turnaround  times  they  would  consider  as  acceptable  for 
different  types  of  examinations  and  what  turnaround  times  they  would 
consider  as  preferred  (giving  the  attending  physician  the  maximum 
benefit  from  radiology  information  in  clinical  decision-making).  The 
consensus  opinions  are  given  in  Appendix  A  for  21  types  of  radiology 
examinations. 

The  decision  to  designate  NHB  as  a  major  evaluation  site  for 
TRIRAD  only  preceded  the  installation  of  the  system  by  a  few  weeks. 
In  order  to  complete  data  collection  activities  within  the  available 
time  and  still  evaluate  the  baseline  operations  in  all  sections  of  the 
Department,  a  decision  was  made  to  focus  the  more  intensive  data 
collection  activities  on  Main  Radiology  and  the  three  imaging  sections 
(Nuclear  Medicine,  Ultrasound,  and  Computerized  Tomography),  which 
together  produced  73Z  of  the  weighted  workload  in  the  Department. 
Smaller  samples  of  data  were  obtained  in  the  other  sections.  The 
TRIRAD  system  itself  made  some  data  easier  to  obtain  in  the  post¬ 
implementation  study,  particularly  data  on  the  timing  of  events  in  the 
radiology-examination-to-report  cycle.  This  made  it  feasible  to 
collect  larger  numbers  of  observations  than  in  the  baseline. 

The  evaluation  measures  and  techniques  are  summarized  in  Table  1. 

Further  detail  concerning  the  scope  of,  and  procedures  followed  in, 

O'* 

evaluation  activities  are  contained  in  the  evaluation  plan. 

C.  DATA  COLLECTION 

Baseline  data  were  collected  during  the  period  November  9  to 
December  4,  1981.  A  total  of  45  man-days  were  spent  on-site 

observing,  collecting  data  and  interviewing  facility  staff. 
Implementation  monitoring  visits  were  conducted  to  NHB  cn  January  20, 
February  16,  March  23,  August  5,  and  October  5,  1982.  Activities 
during  these  visits  included  observation,  interviews,  and  focused 
collection  of  data  on  department  workload  and  other  data  compiled  and 
reported  by  TRIRAD.  Post-implementation  data  were  collected  during 
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the  period  October  18  to  November  5,  1982;  a  total  of  57  man-days  were 
spent  on-site  at  this  time. 

During  the  baseline  study,  a  total  of  78  questionnaires  was 
distributed  and  65  (83X)  were  completed  and  returned.  .This  represents 
a  58X  sample  of  the  staff  working  in  the  Department  at  the  time.  In 
the  post-implementation  study,  questionnaire  administration  was  the 
responsibility  of  the  Department;  95  questionnaires  were  distributed 
and  53  (56X)  completed  questionnaires  were  available  for  analysis. 
This  sample  represents  approximately  47Z  of  the  staff  at  the  time. 
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III.  RADIOLOGY  PROCESS,  VOLUME  AND  EQUIPMENT 

A.  INTRODUCTION 

This  chapter  discusses  the  process  followed  in  delivery  of 
Radiology  services,  including  the  processing  of  patients,  examinations 
and  reports.  The  volume  of  examinations  performed  annually  in  each 
section,  as  well  as  a  list  of  TRIRAD  equipment  within  the  Department, 
is  also  presented. 

The  workflow  for  providing  Radiology  services,  while  varying  in 
detail,  generally  followed  a  consistent  process  in  all  sections.  Two 
generalized  flow  diagrams  have  been  prepared  -  one  each  for  the 
baseline  period  (Figure  1)  and  for  the  post-implementation  period 
(Figure  2).  This  discussion  is  organized  according  to  the  major  steps 
in  the  process:  scheduling  and  reception,  film  file  retrieval,  film 
processing  and  control,  interpretation  and  reporting,  transcription, 
and  report  approval  and  distribution.  Under  each  of  these  subheadings 
the  generalized  procedure  is  presented,  variations  followed  in  each 
section  are  described,  and  differences  between  the  baseline  and 
post-implementation  periods  are  discussed.  The  volume  of  examinations 
and  the  corresponding  weighted  work  units  are  shown  at  the  end  of  this 
Chapter  in  Table  2  for  both  the  baseline  and  post-implementation 
periods,  and  the  peripheral  equipment  for  TRIRAD  located  in  each  area 
of  the  Department  are  shown  in  a  matrix  format  in  Table  3. 

B.  SCHEDULING  AND  RECEPTION 

The  Radiology  process  began  with  the  scheduling  and  reception 
functions.  There  were  five  reception  areas  in  the  Department: 

•  Main  Radiology  reception  provided  scheduling  and 
registration  for  Main  Radiology  and  GI  Radiology; 

e  Imaging  reception  provided  scheduling  and  reception  for 
Nuclear  Medicine,  Computerized  Tomography,  and 
Ultrasonography; 

•  Urology  outpatient  clinic  reception  served  Urologic 
Radiology; 
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•  Orthopedic  outpatient  clinic  reception  served  Orthopedic 
Radiology;  and 

e  The  Special  Procedures  section  scheduled  and  received 
patients,  but  did  not  need  a  reception  desk,  as  such,  due 
to  the  low  volume  of  patients  (three  per  day  on  average). 

During  the  post-implementation  period  all  patients  were  scheduled  on 
the  TRIRAD  system;  some  patients  were  scheduled  in  advance  while 
others  were  scheduled  on  arrival  for  examination.  During  the  baseline 
period  many  patients  were  scheduled  in  advance;  scheduling  information 
was  recorded  in  notebooks  identified  for  that  purpose. 

In  Main  Radiology  patients  were  either  scheduled  for  examination 
in  advance  or  examined  on  a  walk-in  basis.  Advanced  scheduling  was 
done  for  mammograms  by  the  mammography  technician.  The  receptionists 
scheduled  other  examinations  in  advance  for  inpatients  and  for 
outpatients  in  cases  when  the  receptionists  predicted  that  an 
outpatient  would  need  to  wait  too  long  for  examination.  Most  standard 
examinations  were  taken  on  a  walk-in  basis.  Portable  examinations 
were  taken  on  non-ambulatory  patients.  In  GI  Radiology  all  patients 
were  scheduled  in  advance  by  the  GI  technicians. 

Imaging  examinations  were  scheduled  by  both  the  receptionists  and 
technicians.  In  Urologic  Radiology  the  technicians  scheduled  all 
examinations.  Orthopedic  Radiology  provided  all  examinations  on  a 
walk-in  basis.  In  the  Special  Procedures  section  examinations  were 
usually  arranged  by  the  radiologist  in  consultation  with  the  attending 
physician;  scheduling  itself  was  usually  done  by  the  technician. 

The  TRIRAD  system  was  intended  to  have  two  scheduling  options: 
optimal  and  manual  scheduling.  The  optimal  mode,  which  was  not 
operational  before  or  during  the  post-implementation  studv,  was 
designed  to  identify  for  the  scheduler  the  next  available  appointment, 
and  potential  conflicts  with  other  examinations  and  overall  to  assist 
the  scheduler  in  developing  an  optimal  schedule.  The  optimal 
scheduling  module  was  designed  to  prevent  overbooking,  scheduling 
examinations  outside  normal  hours,  and  scheduling  examinations  in 
rooms  where  the  examination  was  not  approved  to  occur. 


The  manual  system  of  scheduling  with  TRIRAD  was  In  use  in  all 
sections  of  Radiology.  The  manual  mode  (which  Is  an  Inappropriate 
label  for  this  automated  scheduling  function)  differed  somewhat  from 
the  optimal  mode.  Examination  times  were  chosen  by  the  scheduler  from 
available  room  schedules  shown  on  the  video  display  terminal.  The 
system  informed  the  operator  of  duplicate  or  similar  examinations, 
conflicts  due  to  contrast  media,  and  conflicts  within  the  patient's 
own  schedule.  In  the  manual  mode,  in  contrast  to  the  optimal  mode, 
the  scheduler  could  choose  to  schedule  an  examination  in  spite  of 
duplicates  or  conflicts.  Overbooking  the  room  schedule,  the  patient's 
schedule  and  scheduling  outside  of  the  established  hours  were  also 
available  in  the  manual  mode. 

When  a  patient  arrived  at  reception  the  arrival  time  was  recorded 
on  the  system  or  In  a  logbook  during  the  baseline  period.  If  the 
patient  was  scheduled  in  advance,  the  prior  film  file  or  a  new  file 
jacket  and  all  identifying  labels  were  waiting  at  the  reception  desk. 
For  unscheduled  patients,  the  patient  was  registered  and  added  to  the 
schedule  and  the  prior  film  file  was  requested  from  the  film  library 
by  submitting  either  a  printed  file  label  or  a  written  note.  The 
printed  labels  replaced  much  of  the  handwriting  required  of 
receptionists  during  the  baseline. 

Inpatients  and  outpatients  were  processed  similarly,  except  that 
unscheduled  inpatients  were  called  from  the  ward  for  examination 
during  low  workload  periods  so  that  waiting  time  was  kept  to  a 
minimum.  Portable  examinations  were  used  for  non-ambulatory 
inpatients,  such  as  those  in  the  Intensive  Care  Unit.  Usually  patient 
registration  and  other  processing  were  done  after  the  examination  in 
order  to  speed  up  the  taking  of  the  film,  especially  for  emergencies. 

After  registration  the  patient  was  asked  to  wait.  For  certain 
emergencies,  preparatory  procedures  were  done  at  this  time,  e.g., 
removing  a  cast  prior  to  a  bone  examination. 

The  differences  between  the  baseline  and  post-implementation 
processes  were  in  the  use  of  the  terminals  to  register  and  schedule 
patients,  the  automated  printing  of  labels  and  lists  of  scheduled 
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examinations!  and  the  Increased  amount  of  patient  information  obtained 
and  the  automated  identification  of  possible  examination  conflicts  and 
duplicates. 

C.  FILM  FILE  RETRIEVAL 

There  were  four  film  libraries:  Main  Radiology,  Imaging, 
Orthopedics,  and  Urology.  The  former  two  were  large  and  contained 
films  for  a  vide  variety  of  examination  types  that  were  needed  by  a 
large  number  of  physicians.  The  latter  two  were  small  and  contained 
specialized  Images  that  were  of  interest  to  a  smaller  group  of 
physicians. 

Films  were  requested  from  the  film  libraries  in  two  ways.  For 
scheduled  patients,  a  list  of  patients  was  submitted  to  the  file  clerk 
during  the  afternoon  or  evening  prior  to  examination.  The  film  files 
were  delivered  to  the  reception  area  to  be  held  until  patient  arrival 
or  a  new  file  jacket  was  prepared.  For  unscheduled  patients, 
individual  requests  or  a  small  group  of  requests  were  submitted.  In 
Main  Radiology  these  requests  were  submitted  on  a  dumbwaiter  to  the 
film  library  on  the  floor  below  reception  and  the  film  files  were 
returned  in  the  same  manner.  The  TRIRAD  functions  that  would  allow 
file  requests  to  be  printed  in  the  film  libraries  were  not  used  (until 
the  final  day  of  post-implementation  data  collection) . 

With  TRIRAD,  when  a  film  could  not  be  located,  data  on  the 
location  of  the  file  were  obtained  and  this  information  was  returned 
to  the  requestor.  Prior  to  releasing  the  film,  the  file  clerk  signed 
out  the  file  to  the  requesting  location. 

D.  FILM  PROCESSING  AND  CONTROL 

After  a  film  was  taken,  it  was  processed  and  matched  with  its 
requisition  and  film  file  jacket  on  the  quality  control  desk.  If  the 
prior  films  had  not  been  received  from  the  film  library,  the  current 
films  remained  on  the  control  desk  until  they  were  matched  (during 
post-implementation)  or  were  sent  to  the  reading  rooms  for 
interpretation  (during  the  baseline  or  for  emergency  patients  during 
the  post-implementation  period) . 
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Once  matched,  films  were  reviewed  for  quality  of  the  images  and, 
if  acceptable,  transferred  to  the  reading  room.  Occasionally  repeat 
examinations  or  additional  views  were  necessary  to  Improve  the  quality 
of  the  examinations.  In  some  sections  films  were  also  reviewed  by  the 
radiologist  in  order  to  ensure  that  the  films  clearly  showed  the 
diagnostic  information  needed.  In  some  sections  the  radiologist  also 
conducted  or  participated  in  the  examination. 

After  films  had  been  accepted  for  quality,  patients  were 
released,  and,  during  post-implementation  "departed"  on  the  TRIRAD 
terminal.  Films  were  transferred  to  a  reading  room.  If  a  wet  reading 
(urgent  interpretation)  was  required,  the  patient  waited  for  the 
report. 

E.  INTERPRETATION  AND  REPORTING 

Interpretation  and  reporting  varied  in  each  section  of  Radiology 
and  in  some  sections  differed  between  baseline  and  post-implementation 
periods. 

Main  Radiology  had  three  reading  areas  -  the  Wet  Reading  Room, 
the  Inpatient  Reading  Room,  and  the  CCU  Reading  Room.  During  the 
baseline  period,  the  Wet  Reading  Room  processed  most  of  the  volume  in 
this  section.  Reports  for  both  inpatients  and  outpatients  were 
handwritten  on  the  requisition  and  signed  as  official  final  reports. 
Hot  all  films  required  wet  readings,  but  all  were  handwritten  In  order 
to  avoid  long  delays  associated  with  the  transcription  of  dictated 
reports.  In  the  post-implementatior.  period  only  outpatient 
examinations  and  inpatient  examinations  requiring  wet  readings  were 
interpreted  in  this  reading  room  and  nearly  all  reports  were  dictated 
for  transcription  or  were  prepared  using  bar-coded  reporting 
assistance.  When  a  wet  reading  report  was  required,  a  preliminary 
handwritten  report  was  prepared  and  a  copy  was  given  to  the  patient 
for  distribution  to  the  requesting  provider.  During  the  baseline 
period,  the  Wet  Reading  Room  was  used  substantially  for  teaching 
Radiology  residents,  but  this  was  not  the  case  during  the 
post-implementation  period. 


The  Inpatient  Reading  Room  served  two  functions  -  as  a  reading 
room  and  as  an  active  storage  area  for  viewing  current  inpatient 
films;  films  were  mounted  and  available  for  viewing  throughout  the 
patient's  stay.  During  the  baseline  period,  this  room  served 
primarily  as  a  film  viewing  room  as  only  a  small  volume  of  films  were 
interpreted  here;  these  reports  were  all  dictated  for  transcription. 
During  the  post-implementation  period,  nearly  all  Inpatient  films  were 
interpreted  in  this  area;  reports  were  all  dictated  for  transcription. 

The  CCU  Reading  Room  operated  similarly  to  the  Inpatient  Reading 
Room;  however,  a  smaller  volume  of  films  were  interpreted  in  this 
area. 

In  GI  Radiology  the  films  for  each  morning's  examinations  were 
mounted  on  a  film  viewer  and  interpreted  during  the  early  part  of  each 
afternoon.  The  films  were  left  for  viewing  for  I  day.  Reading 
sessions  were  also  teaching  sessions  for  residents.  During  these 
sessions  the  residents  took  handwritten  notes,  which  were  written  as 
preliminary  reports  in  a  notebook;  the  preliminary  reports  were 
available  to  providers  when  they  came  to  view  the  Cl  films.  The 
residents  dictated  the  final  interpretation  for  transcription.  The 
process  was  the  same  in  both  study  periods. 

In  Orthopedic  Radiology,  all  reports  were  treated  as  wet 
readings;  a  final  handwritten  interpretation  was  available  within 
minutes  after  the  films  were  processed.  These  reports  were 
distributed  Immediately  to  the  orthopedic  surgeons.  During  the 
post-implementation  period,  this  practice  continued,  but  reports  were 
also  entered  into  the  system  to  create  the  historical  file;  a  small 
volume  was  entered  by  bar  coding  and  the  major  share  of  the  volume  by 
transcription  from  the  handwritten  reports. 

In  Urologic  Radiology,  Special  Procedures,  Nuclear  Medicine  and 
Computerized  Tomography  all  films  were  dictated  for  transcription 
during  both  periods.  Nuclear  Medicine  began  a  practice  of  direct 
dictation  to  a  transcriptlonist  for  simultaneous  typing  in  the  middle 
of  the  post-implementation  data  collection  period.  In  Ultrasonography 
reports  were  dictated  for  transcription  during  the  baseline  period  and 
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these  reports  were  interpreted  in  reading  sessions.  During  the 
post-implementation  period,  reports  were  interpreted  immediately  on 
completion  of  the  examination  and  these  handwritten  reports  were  sent 
to  transcription. 

F.  TRANSCRIPTION 

During  the  baseline  period,  dictated  final  reports  and  the 
related  requisitions  were  sent  by  Main  Radiology,  GI  Radiology, 
Urologic  Radiology  and  Special  Procedures  to  the  hospital's  Central 
Transcription  service  located  in  Patient  Affairs.  In  Patient  Affairs, 
the  envelopes  were  delivered  to  the  one  transcriptionist  assigned  to 
Radiology  transcription.  This  transcriptionist  recorded  the  date 
received  on  the  envelope  and  added  it  to  the  queue  of  work  to  be 
processed. 

The  transcriptionist  typed  the  dictated  reports  from  the  cassette 
onto  the  requisitions  that  were  included  with  the  cassette.  Completed 
reports  were  Inserted  back  into  the  envelope  and  set  aside.  At  day’s 
end  all  the  envelopes  containing  the  completed  typed  dictations  were 
put  in  the  outgoing  mail  for  delivery  back  to  Radiology. 

The  process  of  selecting  the  order  in  which  the  envelopes  are 
processed  appeared  to  be  made  without  regard  to  date  of  receipt  or 
date  of  examination.  The  shorter  reports  were  generally  typed  first, 
especially  chest  X-ray  reports. 

Patient  Affairs  contracted  with  an  outside  private  transcription 
service  to  supplement  the  staff  at  hand.  As  the  queue  of  incomplete 
dictations  grew  so  did  the  likelihood  that  they  would  be  sent  to  the 
outside  service.  Long  reports,  especially  angiograms,  were  more 
likely  to  be  sent  out  to  a  contract  typing  service.  The  outside 
transcription  service  typed  the  dictated  final  report  onto  the 
■ncluded  requisitions  and  returned  them  to  Patient  Affairs,  which 
distributed  them  to  Radiology,  as  described  above. 

On  receipt  of  the  typed  reports.  Radiology  distributed  them  for 
signature.  The  signed  reports  were  separated  and  the  copies  were  sent 
to  the  requesting  physician,  the  outpatient  clinic  or  inpatient  ward, 
and  the  film  file  folder  that  contained  the  patients'  X-rays. 
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The  Imaging  sections  of  Radiology  did  not  use  the  Patient  Affairs 
transcription  services  so  the  processing  of  final  dictated  reports 
differed  from  that  described  above  for  the  other  sections  of 
Radiology.  Both  in-house  and  contract  typing  services  were  used. 

Dictated  Nuclear  Medicine  reports  were  delivered  to  the  Nuclear 
Medicine  secretary  for  transcription.  Because  of  the  relatively  low 
volume  of  reports,  this  secretary  was  able  to  complete  the  reports  in 
addition  to  her  other  departmental  responsibilities.  Completed 
reports  were  distributed  for  signature  and,  at  weekly  intervals,  were 
distributed  to  requesting  physicians,  to  clinics  and  wards  and  to 
file. 

Dictated  reports  for  Ultrasonography  and  Computerized  Tonography 
were  delivered  to  the  Nuclear  Medicine  secretary  for  processing. 
Because  the  Nuclear  Medicine  secretary  only  typed  the  Nuclear  Medicine 
reports  and  the  Patient  Affairs  transcription  services  had  a  lengthy 
turnaround  time,  an  alternative  transcription  service  was  sought. 

In  August  1981  the  backlog  of  untyped  Ultrasonography  and  CT 
reports  had  become  considerable.  After  bidding  and  contractual  terns 
were  arranged,  several  hundred  reports  were  sent  out  for  transcrip¬ 
tion.  During  the  baseline  data  collection  period  of  November  1981, 
the  same  contract  services  were  used  based  on  the  previously 
established  terms.  The  transcription  process  for  Ultrasound  and  CT 
reports  was  apparently  being  changed  and  in  transition  at  the  time  of 
data  collection. 

Typed  reports  were  returned  to  the  Radiology  supply  officer  and 
distributed  for  signature  through  the  office  of  the  Nuclear  Medicine 
secretary.  Signed  reports  were  separated  and  copies  were  sent  to  the 
requesting  physician,  the  patient  clinic  or  inpatient  location  and  to 
the  file. 

In  the  post-implementation  period,  the  transcription  process  was 
completely  revised  and  all  transcribed  reports  vere  entered  into  the 
computer.  Two  transcriptionists  were  added  to  the  Radiology  staff, 
and  typing  of  Radiology  reports  was  no  longer  done  by  Central 
Transcription  or  contract  typing  services.  The  Nuclear  Medicine 
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secretary  usually  transcribed  reports  from  that  section  and  did  both 
transcription  from  direct  dictation  and  from  dictation  tape.  In 
Orthopedic  Radiology  and  Ultrasonography  reports  were  transcribed  from 
handwritten  preliminary  reports.  For  all  other  sections,  reports  were 
transcribed  from  dictation  tape.  The  pre-defined  text  feature  of 
TRIRAD  was  used  with  the  goal  to  increase  the  efficiency  of  both 
interpretation  and  transcription.  Using  this  feature,  the  radiologist 
dictated  a  code  for  an  interpretation  statement  which  was  pre-defined 
and  stored.  The  transcriptionist  then  retrieved  this  statement  by 
typing  only  the  code  on  the  video  display  terminal.  Once  the  reports 
had  been  entered  into  the  system,  they  were  available  for  radiolo¬ 
gists'  approval. 

G.  REPORT  APPROVAL  AND  DISTRIBUTION 

During  the  baseline  period,  handwritten  final  reports  were  signed 
and  distributed  immediately.  For  wet  readings,  a  copy  was  given  to 
the  patient  for  distribution  to  the  provider.  For  routine  reports, 
the  distribution  copies  were  decollated  and  inserted  into  the  mailbox 
for  distribution  to  medical  records,  attending  physicians  and  to  the 
film  files.  Typed  reports  were  delivered  to  the  radiologists  for 
review  and  signature  and  then  these  were  decollated  and  inserted  into 
the  mailbox  for  distribution. 

During  the  post-implementation  period,  handwritten  reports  were 
signed  and  distributed  immediately.  Reports  entered  into  the  system 
by  bar  coding  or  by  transcription  were  approved  by  radiologists  using 
electronic  signature.  After  reviewing  the  report  and  editing  the  text 
as  needed  using  an  intelligent  video  display  terminal,  the  radiologist 
approved  the  report  with  a  unique  signature  code.  These  reports  were 
then  transferred  to  a  print  queue.  Several  times  each  day  a 
designated  technician  printed  all  reports  in  this  queue.  The  reports 
were  accumulated  and  then  later  decollated  and  inserted  in  the  mailbox 
for  distribution  to  the  ward  or  clinic,  medical  record,  the  attending 
physician  and  to  the  film  file. 
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H.  TRIRAD  PERIPHERAL  EQUIPMENT 


A  matrix  of  Che  TRIRAD  peripheral  equipment  units  that  are 
located  in  each  area  of  Radiology  is  shown  in  Table  2.  These  units 
include  video  display  terminals,  bar-code  readers  and  printers  as 
indicated,  but  not  the  mainframe  equipment  and  software.  The  numbers 
in  the  columns  refer  to  the  numbers  of  each  unit  located  in  each  area 
of  Radiology.  For  example,  there  are  three  video  display  terminals  in 
the  Main  Radiology  reception  area.  For  additional  information  on  the 
mainframe  and  software,  refer  to  the  section  on  Costs  in  Chapter  T.V 
and  to  Appendix  E. 

I.  VOLUMES  AND  WEIGHTED  WORKLOAD 

The  annual  FY81  and  FY82  volumes  of  examinations  performed  by 
each  of  the  Radiology  sections  are  presented  in  Table  3.  The  percent 
of  the  total  volume  in  each  section  has  also  been  computed.  Radiation 
Therapy  treatments  were  not  included  since  this  section  did  not  use 
TRIRAD  as  of  November  of  1982.  Volumes  for  FY81  have  been  used  as 
baseline  values  since  it  was  Che  last  full  year  prior  to  the 
Installation  of  TRIRAD.  Similarly,  FY82  volumes  have  been  used  for 
post-implementation  since  the  first  year  of  TRIRAD  operations  began  in 
the  second  quarter  of  FY82,  although  TRIRAD  was  actually  used  for  only 
9  months  of  FY82. 

Also  presented  in  Table  3  are  the  annual  workload  values  reported 
as  weighted  volumes  for  FY81  and  FY82.  Again,  the  percentage  of  total 
workload  attributable  to  each  section  is  provided.  The  weighted  value 
is  a  quantitative  method  used  to  account  for  the  differences  in 
resources  required  for  each  type  of  examination.  For  example,  a 
simple  chest  X-ray  has  a  weighted  value  of  3  whereas  an  angiogram, 
which  requires  a  number  of  physicians  and  technicians,  a  well-equipped 
special  procedures  room  and  costly  supplies,  has  a  weighted  value  of 
28.  All  examinations  have  an  average  weighted  value  of  5.3.  It 
should  be  noted  that  the  method  for  computing  weighted  workload  values 
for  Nuclear  Medicine  changed  in  FY82.  During  the  baseline  period  each 
examination  was  assigned  an  average  weighted  value  of  10;  during  the 
post-implementation  period  the  average  weighted  value  was  57, 
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TABLE  2 


PERIPHERAL 

EQUIPMENT  CONFIGURATION 

EVALUATION  OF 

TRIRAD 

NAVAL 

HOSPITAL, 

BETHESDA 

Number  of 

Devices 

VDT 

IVDT 

BCR 

BCP 

PTR 

Hospital  Main  Computer  Room 

1 

la 

Main  Radiology 

1 

Reception 

3 

Quality  Control 

I 

I 

ICU/CCU  Readlong  Room 

1 

Wet  Reading  Room 

1 

2 

1 

Inpatient  Reading  Room 

1 

5 

Administrative  Offices 

1 

1 

2 

2 

Main  Film  Library 

1 

2 

1 

GI  Radiology 

1 

1 

Quality  Control 

Reading  Room 

1 

Imaging  Radiology 

1 

1 

Reception 

Reading  Room 

2 

1 

2 

CT  Image  Processing  Room 
Administrative  Offices 

2 

I 

1 

1 

File  Room 

1 

Orthopedic  Radiology 

1 

Reading  Room 

1 

1 

File  Room 

1 

1 

Urologic  Radiology 

I 

1 

Reception 

I 

Reading  Room 

File  Room 

1 

1 

Special  Procedures 

1 

1 

1 

Reading  Room 

Total 

17 

4 

24 

7 

6 

aOne  bar-code  printer,  originally  assigned 

to  Radiation  Therapy,  is 

used 

back-up  to  substitute  when  one  is  being  maintained 

or  repaired. 

Legend : 

VDT  -  video  display  terminal 

BCR 

bar-code  printer 

IVDT  -  intelligent  video  display  terminal 

PTk  - 

printer 

BCR  -  bar-code  reader 

OMR  - 

optical-mark  reader 

COMPARISON  OF  BASELINE  AND  POST-IMPLEMENTATION  ANNUAL  RADIOLOGY  VOLUMES  AND  WEIGHTED  WORKLOAD  VALUES 

EVALUATION  OF  TRIRAD 
NAVAL  HOSPITAL,  BETHESDA 
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In  Nuclear  Medicine  Che  method  for  computing  weighted  workload  value  changed.  Whereas  during  the  baseline  period  each 
xaalnation  was  assigned  an  average  weighted  value  of  10,  during  the  post-implementation  period  the  average  weighted 


The  workload  figures  that  are  displayed  In  Table  3  were  obtained 
from  data  provided  by  Uniform  Chart  of  Accounts  (UCA)  reports. 
Radiology  department  records,  and  interviews  with  members  of  the 
Radiology  staff  and  staff  In  the  NHB  Comptroller's  office.  The 

apparent  increase  in  the  total  number  of  examinations  from  FY8I  to 
FY82  (FY81  -  84,574;  FY82  -  92,816)  may  be  due  at  least  partially  to 
Radiology's  increased  capability  to  capture  and  record  examinations  by 
using  the  TRIRAD  system  and  may  not  fully  represent  an  actual  Increase 
in  volume.  (Staff  in  the  Department  believed  that  the  Wet  Reading 
Room  in  Main  Radiology,  where  a  large  volume  of  handwritten  reports 
«  was  processed  during  Che  baseline  period,  had  experienced  difficulties 

in  tracking  completed  examinations  and  that  this  vas  a  mejcr 
contributor  to  the  apparent  differences  in  reported  volume.)  The 
«  change  in  weighted  workload  is  due  in  part  to  the  increase  in 

examination  volume  and  to  the  change  in  weighting  procedures  as 
discussed  above. 

The  Incomplete  reports  data  presented  in  Chapter  IV  present 
supporting  data  to  further  evaluate  the  validity  of  the  changes  in 
volume  observed.  For  a  more  detailed  view  of  post- implementation 
volumes,  see  Appendix  F  which  presents  the  daily  examination  volumes 
for  each  Department  section. 
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IV.  RESULTS 


A.  INTRODUCTION 

This  chapter  presents  the  results  of  the  various  evaluation 
activities.  Results  of  quantitative  studies  are  presented  first, 
organized  by  the  impacts  studied  as  follows: 

e  report  turnaround  time; 

•  patient  process  time; 

e  labor  to  process  patients,  reports,  films;  and 

•  film  file  availability. 

The  next  section  contains  the  information  gathered  from  Department 
records,  TRIRAD  contract  documents,  and  interviews  concerning  the 
costs  of  radiology  services.  The  final  section  first  analyzes  the 
results  of  baseline  and  post-implementation  surveys  of  Radiology 
Department  staff  and  then  summarizes  the  Information  gained  from 
baseline  and  post-implementation  interviews  with  attending  physicians. 
Chapter  V  contains  a  discussion  of  results  combining  the  results  of 
quantitative  and  qualitative  studies,  and  interprets  these  results  in 
reference  to  system  goals  and  evaluation  objectives. 

B.  QUANTITATIVE  STUDIES 

1.  Turnaround  Time  for  Radiology  Reports 

a.  Introduction 

Two  types  of  data  were  collected  regarding  the  turnaround  tine 
for  radiology  reports.  The  first  was  a  measure  of  the  elapsed  time 
for  the  complete  processing  of  patient  examinations  from  the  time  the 
patient  arrived  in  the  Department  for  the  examination  until  the 
interpretation  results  were  available  for  distribution  to  the 
attending  physician  and/or  the  patient  record.  The  second  measured 
the  ability  of  the  Department  to  produce  reports  through  the  process 
by  examining  the  backlog  of  examinations  for  which  interpretation 
reports  had  not  been  completed  (incomplete  reports). 
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b.  Elapsed  Time  Analysis 

1.  Data  Considerations 

The  elapsed  time  was  measured  from  che  time  of  patient  arrival 
for  the  examination  until  the  interpretation  report  was  available  for 
distribution  via  the  department  mailbox  or  (for  wet  readings)  by  hand 
delivery  by  the  patient.  Interpretation  reports  were  distributed  in 
several  different  forms.  In  the  baseline,  handwritten  reports  and 
reports  typed  from  dictation  were  prepared.  In  the  post-implemen¬ 
tation  process,  most  reports  were  printed;  however,  handwritten 
reports  were  also  prepared  for  those  examinations  for  which  a  wet 
reading  had  been  requested.  Elapsed  times  were  evaluated  for  all  of 
these  report  types.  In  addition,  in  Main  Radiology,  the  process  after 
the  implementation  of  TRIRAD  differed  for  inpatients  and  outpatients 
and  both  of  these  were  evaluated  separately. 

Elapsed  times  were  studied  for  the  whole  process  and  for 
different  segments  of  the  process. 

For  typed  reports  in  the  baseline  period,  five  time  segments  were 
either  measured  or  estimated  as  follow: 

e  time  segment  from  examination  to  completion  of  dictation; 

e  time  segment  from  dictation  to  delivery  of  the 
dictation  tape  to  transcription; 

•  time  segment  from  delivery  to  transcription  until 
completion  of  typing; 

•  time  segment  from  completion  of  typing  until  report 
returned  to  Radiology;  and 

•  time  segment  from  arrival  in  Radiology  until 
report  approved  and  distributed. 

For  printed  reports  in  the  post-implementation  period,  two  time 
segments  were  measured  as  follow: 

•  time  segment  from  examination  through  generation 
of  the  computer  report  (this  interval  contained 
dictation  and  transcription,  or  bar  coding,  as 
well  as  report  approval) ;  and 

•  time  segment  from  report  printing  through  distribution 
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into  the  mailbox. 

For  handwritten  reports,  report  turnaround  tine  was  measured  in 
two  segments  as  follow: 

e  time  segment  from  patient  arrival  in  Radiology, 
through  examination  and  patient  departure;  and 

•  the  time  segment  spent  in  reading  the  film  and 
preparing  the  report. 

Data  were  collected  in  a  different  manner  in  the  baseline  and 
post-implementation  studies.  In  the  baseline,  data  were  difficult  to 
obtain  on  the  entire  cycle  because  the  elapsed  time  for  many  reports 
exceeded  the  length  of  the  data-collection  period  and  data  were 
captured  by  a  variety  of  techniques  for  different  segments  of  the 
cycle,  with  the  total  elapsed  time  derived  from  the  available  data. 
In  the  post-implementation  period,  many  of  the  data  which  needed  to  be 
derived  in  the  baseline  study  could  be  obtained  from  the  TRIRAD 
system,  and  this  made  it  possible  to  obtain  more  observations  and  with 
greater  consistency  across  sections  of  the  department  and  process 
segments  and  to  track  Individual  examinations  to  completion.  Hence, 
there  is  necessarily  greater  confidence  in  the  reliability  of 
post-implementation  data  concerning  this  impact. 

ii.  Results 

A  comparison  of  measured  report  turnaround  times  is  contained  in 
Table  4.  In  every  section  of  the  Department  turnaround  time  for  typed 
reports  improved  in  the  post-implementation  period  as  compared  with 
baseline  observations.  For  handwritten  reports,  turnaround  time 
increased  slightly. 

The  turnaround  time  in  Main  Radiology  decreased  from  an  estimated 
16  days  in  the  baseline  period  for  typed  reports  to  4  days  for  printed 
reports  on  both  inpatients  and  outpatients  in  the  post-implementation 
period.  For  a  limited  sample  of  four  bar-coded  reports,  turnaround 
time  was  measured  as  2  days.  In  the  baseline,  turnaround  for 
handwritten  reports  averaged  46  minutes.  During  post-implementation 
handwritten  reports  were  prepared,  in  addition  to  the  printed  reports. 
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for  examinations  on  which  a  "wet  reading"  was  requested;  turnaround 
time  for  these  reports  averaged  63  minutes. 

The  proportion  of  reports  prepared  in  handwritten  versus 
typed/printed  form  changed  significantly  between  the  baseline  and 
post-implementation  periods.  Table  4  combines  data  on  volume  changes 
and  elapsed  time  for  report  turnaround.  As  shown,  nearly  all 
(approximately  952)  of  reports  produced  in  the  baseline  period  were 
handwritten,  whereas  in  the  post-implementation  period,  relatively  few 
handwritten  reports  were  prepared,  with  many  of  these  serving  as 
preliminary  reports  for  wet  reading  examinations. 

The  reports  for  GI  Radiology  were  dictated  and  typed  and 
distributed  during  the  baseline  period  in  an  average  time  of  18  days. 
This  time  was  reduced  to  an  average  of  5  days  for  computer-printed 
reports. 

In  Orthopedic  Radiology  all  reports  were  handwritten  during  the 
baseline  period  and  distributed  in  an  average  time  of  39  minutes. 
During  the  post-implementation  period,  handwritten  reports  were 
prepared  in  the  same  manner  in  an  average  time  of  56  minutes.  In 
addition  a  copy  of  the  report  was  delivered  to  transcription  to  be 
entered  into  the  TRIRAD  system.  The  printed  report  was  distributed  in 
an  average  of  4  days  from  the  date  of  examination. 

Decreased  turnaround  times  were  measured  in  all  other  sections 
(Table  4).  In  Urologic  Radiology,  report  turnaround  time  decreased 
from  an  estimated  18  days  to  5  days  (measured  on  a  very  small  sample 
of  reports) .  In  Special  Procedures  turnaround  time  was  reduced  from 
an  estimated  25  days  to  14  days;  in  Nuclear  Medicine  from  15  days  to 
12  days;  in  Computerized  Tomography  from  43  days  to  14  days;  and  in 
Ultrasonography  from  34  days  to  6  days. 

Figure  3  illustrates  the  relative  contribution  of  individual 
process  segments  to  total  turnaround  time.  In  the  baseline  period  the 
time  segment  in  transcription  contributed  to  delays  in  all  sections, 
except  in  Orthopedic  Radiology  where  only  handwritten  reports  were 
prepared.  Other  major  delays  were  also  observed  in  Computerized 
Tomography  for  time  segments  from  examination  through  dictation,  and 
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from  dictation  through  delivery  to  transcription.  In  Ultrasonography, 
delays  were  noted  in  the  time  segment  from  dictation  through  delivery 
to  transcription. 

In  the  post-implementation  period,  the  time  segment  from 
examination  to  report  averaged  3.5  days,  the  time  segment  from  report 
printing  to  distribution  in  the  mail  averaged  2.5  days,  and  the  total 
time  across  all  sections  averaged  6  days.  (For  a  limited  sample  of 
four  bar-coded  reports,  each  report  was  printed  on  the  same  day  as  the 
examination  and  distributed  to  the  mailbox  in  2  days,  resulting  in  a 
report  turnaround  time  of  2  days.) 

During  the  post-implementation  period,  in  order  to  evaluate 
better  the  implications  of  a  6-day  turnaround  time  for  official 
Radiology  reports  on  inpatient  care,  a  determination  was  made  of  the 
number  of  inpatients  still  in-house  when  Radiology  reports  were 
distributed.  As  part  of  the  data-collection  procedures  for  measuring 
turnaround  time  at  the  mailbox,  patient-identification  data  were  also 
obtained.  When  reports  were  picked  up  for  distribution  to  inpatient 
wards,  a  comparison  was  made  by  patient-ID  with  the  inpatient  census. 
The  number  of  patients  still  in-house  on  the  day  of  distribution  was 
found  to  be  33%.  Since  some  patients  also  check  out  of  the  hospital 
on  that  day  and  the  report  might  not  be  examined  prior  to  patient 
departure,  the  census  on  the  following  day  was  also  compared;  it  was 
found  that  26%  of  inpatients  were  still  in-house  on  the  day  following 
report  distribution.  A  similar  study  was  not  performed  during  the 
baseline  period. 

c.  Complete  and  Incomplete  Reports 

1.  Data  Considerations 

The  numbers  of  complete  and  incomplete  reports  were  analyzed  from 
management  reports  generated  by  the  TRIRAD  system.  A  complete  report 
is  one  that  has  been  entered  into  the  system,  approved  by  the 
radiologist,  and  then  printed.  An  incomplete  report  is  one  that  has 
not  completed  this  cycle;  thus  a  dictated  report  that  has  been  entered 
onto  the  system  by  the  transcriptionist  but  has  not  yet  been  approved 
is  considered  incomplete. 
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The  incomplete-report  analysis  was  conducted  in  order  to 
supplement  the  analysis  of  elapsed  time  to  report  distribution. 
Turnaround  time,  having  been  measured  at  the  distribution  mailbox, 
tracks  Chose  reports  that  complete  the  cycle  and  does  not  indicate  the 
backlog  of  examinations  for  which  a  report  has  not  been  generated. 
The  incomplete  report  analysis  was  made  possible  by  TRIRAD  for  each 

*  section  of  the  Radiology  Department,  the  total  number  of  procedures 
conducted  on  each  day  and  the  distribution  of  complete  and  incomplete 
reports  for  each  day. 

The  computer  reports,  from  which  these  data  were  obtained,  were 
1  generated  on  four  different  dates;  three  during  site  visits  to  monitor 

the  progress  of  implementation  on  March  23,  August  5  and  October  5  and 
a  fourth  at  the  conclusion  of  post-implementation  data  collection  on 

►  November  5,  1982.  A  comparison  reveals  trends  in  the  report  backlog 

of  the  Department  and  the  individual  sections. 

Information  on  the  status  of  report  completion  was  not  available 
in  the  baseline  nor  was  it  feasible  to  collect  such  information. 
However,  the  first  sampling  date  of  March  23  was  only  2  months  after 
the  installation  of  TRIRAD,  when  the  Department  was  still  in  the 
preliminary  stages  of  implementing  the  system  into  routine  operations. 
Furthermore,  the  Information  on  report  completion  for  that  date 
indicates  a  mean  report  completion  time  of  approximately  16  days. 
Since  this  is  consistent  with  the  measured  mean  elapsed  time  for 
baseline  report  turnaround,  report  completion  Information  for  that 
sampling  data  would  appear  to  represent  a  reasonable  analog  to  the 
baseline. 

li.  Results 

Figure  4  illustrates  the  status  of  complete  and  incomplete 
reports  for  all  Radiology  sections  combined  as  of  each  of  the  sampling 
dates.  The  low-volume  periods  or  the  "valleys"  occur  on  weekend  days 
and  the  high-volume  periods  or  "peaks"  occur  on  weekdays. 

The  following  example  explains  how  to  interpret  these  graphs.  On 

*  March  23,  a  printout  was  obtained  indicating  the  status  of  reports  for 
the  month  preceding  that  day.  On  the  preceding  day,  March  22, 
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FIGURE  4  POST-IMPLEMENTATION  RADIOLOGY  REPORT  COMPLETION  TRENDS, 
ALL  RAOIOLOGY  SECTIONS  -  MONTH  PRECEDING  MARCH  23, 1982, 
AUGUST  5, 1982,  OCTOBER  5,  1982  AND  NOVEMBER  5, 1982  - 
EVALUATION  OF  TRIRAD,  NAVAL  HOSPITAL,  BETHESDA 


approximately  350  examinations  were  completed.  Approximately  320  of 
these  were  Incomplete  and  30  were  completed  by  March  23.  Sixteen  days 
prior  to  March  23,  on  March  7,  there  were  approximately  350 
examinations  and  of  these  20C  reports  were  Incomplete  and  150  reports 
were  complete  on  March  23.  As  more  days  pass,  more  reports  have  been 
completed  for  examinations  on  a  given  date. 

Comparing  March  23  with  November  5,  substantial  changes  are  noted 
in  the  Department's  ability  to  complete  reports.  In  the  November  5 
data,  of  approximately  290  examinations  performed  on  preceding  day  3, 
200  of  these  were  completed  by  November  5;  this  finding  is  consistent 
with  the  average  elapsed  time  of  3  days  cited  above  for  the  time 
segment  between  examination  and  report  printing.  Though  most  reports 
were  completed  within  several  days,  there  still  remained  approximately 
50  incomplete  reports  for  each  weekday,  i.e.,  15  percent  of  the  total 
volume  of  examinations,  even  a  month  later.  A  certain  portion  of 
these  were  never  meant  to  have  official  reports,  e.g.,  procedures 
performed  as  part  of  a  research  study.  Another  explanation  is  that 
some  examinations  were  reported  by  methods  other  than  by  using  the 
TRIRAD  system. 

Graphs  of  complete  and  incomplete  reports  for  each  of  the 
individual  sections  are  contained  in  Figures  5  through  11.  The  data 
for  Main  Radiology  (Figure  5)  reveal  a  similar  pattern  of  improvement 
between  March  and  November  as  was  described  above  for  the  total 
department,  (Note  that  these  graphs  of  Main  Radiology  also  contain  GT 
studies. ) 

The  data  for  Orthopedic  Radiology  (Figure  6)  have  to  be 
interpreted  differently  from  those  for  the  other  sections.  Since  all 
examinations  were  treated  as  wet  readings,  handwritten  reports  were 
prepared  for  immediate  distribution.  Reports  were  also  entered  into 
the  system  in  order  to  maintain  an  historic  report  file  on  each 
patient.  In  March  very  few  reports  were  entered,  whereas  in  November 
most  reports  were  entered  and  completed  within  2  days  of  the 
examination  date. 
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In  Urologic  Radiology  (Figure  7),  all  procedures  were  logged  on 
che  TRIRAD  system,  despite  the  fact  that  a  portion  of  them  were  not 
radiologic  procedures  and  not  Intended  to  have  radiologic  reports. 
Hence,  Improvements  In  completing  reports  are  apparent,  but  not  to  the 
degree  measured  In  other  sections. 

The  low  volume  In  Special  Procedures  (Figure  8)  makes  changes 
difficult  to  appreciate.  Rapid  completion  of  reports  was  observed  in 
August,  worsened  somewhat  in  October,  but  improved  again  in  November 
which  appeared  to  be  consistent  with  staffing  changes. 

Nuclear  Medicine  (Figure  9)  remained  fairly  constant  in 
completing  reports,  worsening  slightly  in  October  and  recovering  again 
in  November.  Computerized  Tomography  (Figure  10)  substantially 
decreased  the  tine  to  complete  reports  between  March  and  August, 
increased  in  October,  and  had  made  gains  in  November.  Ultrasonography 
(Figure  11)  steadily  improved  from  August  to  October  to  November. 

2.  Labor  and  Process  Time 

a.  Introduction 

Data  were  collected  in  both  baseline  and  post-implementation 
periods  pertaining  to  Radiology  Department  staff  time  required  to 
carry  out  specific  functions  and  elapsed  times  for  selected  segments 
of  the  process.  Labor  times  were  measured  for  the  following 
functions: 

e  Time  devoted  to  scheduling/reception  activities  in 
reception  areas; 

e  Time  devoted  to  radiologist  interpretation  and  reporting; 
and 

e  Time  devoted  to  transcribing  reports. 

Elapsed  times  were  measured  for  selected  segments  of  the  process  as 
follow: 

e  Patient  waiting  and  examination  time; 

e  Elapsed  time  for  film  file  requests;  and 

e  Elapsed  time  from  dictation  until  completion  in 

transcription. 
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FIGURE  V  POST-IMPLEMENTATION  RADIOLOGY  REPORT  COMPLETION  TRENDS. 
SPECIAL  PROCEDURES  SECTION  -  MONTH  PRECEDING  MARCH  23, 
1982,  AUGUST  5, 1982,  OCTOBER  6.  1982  AND  NOVEMBER  6,  1982  - 
EVALUATION  OF  TRIRAD,  NAVAL  HOSPITAL,  BETHESDA 


FIGURE  8  POST  IMPLEMENTATION  RADIOLOGY  REPORT  COMPLETION  TRENDS, 
ULTRASONOGRAPHY  SECTION  -  MONTH  PRECEDING  MARCH  23, 1982, 
AUGUST  5, 1982,  OCTOBER  5,  1982  AND  NOVEMBER  5,  1982  - 
EVALUATION  OF  TRIRAD,  NAVAL  HOSPITAL.  BETHESDA 


FIGURE  9  POST-IMPLEMENTATION  RADIOLOGY  REPORT  COMPLETION  TRENDS, 
COMPUTERIZED  TOMOGRAPHY  SECTION  -  MONTH  PRECEDING 
MARCH  23,  1962,  AUGUST  5, 1982,  OCTOBER  6, 1982  AND  NOVEMBER  5, 
1982  -  EVALUATION  OF  TRIRAD,  NAVAL  HOSPITAL,  BETHESDA 
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The  results  of  these  are  discussed  below  In  the  order  that  the 
steps  occur  In  the  Radiology  process. 

b.  Tine  Devoted  to  Scheduling/Reception 

1.  Data  Considerations 

Work  sampling  was  conducted  during  baseline  and 
post-implementation  periods  in  the  reception  areas  of  Main  Radiology 
and  Imaging.  The  specific  activities  sampled  were  those  pertaining  to 
scheduling  and  check-in  functions,  with  the  purpose  of  ascertaining 
any  change  in  the  proportion  of  staff  time  devoted  to  these 
activities.  The  activities  observed  varied  substantially  because  of 
the  changes  Introduced  by  TRIRAD. 
li.  Results 

The  results  of  work  sampling  are  shown  in  Tables  5  and  6  for  the 
baseline  and  post-implementation  periods,  respectively.  Comparison  of 
the  two  periods  shows  some  major  changes  in  the  activities  observed. 
There  has  been  an  Increase  in  the  proportion  of  time  spent  in  working 
directly  on  scheduling  and  reception  functions,  noted  as  a  decrease  in 
"other"  time  at  both  reception  areas.  In  Imaging  "other"  time 
decreased  from  40.2%  in  the  baseline  period  to  7.3%  during  the 
post-implementation  study.  In  Main  Radiology  "other"  time  changed 
from  47.1%  down  to  30.0%. 

During  post-implementation,  computer  functions  occupied  32.1%  of 
the  receptionist's  time  in  Imaging  and  28.2%  of  the  time  in  Main 
Radiology.  The  percent  of  time  devoted  to  scheduling  and  receiving 
patients  was  similar  in  both  sections  during  the  baseline  period  — 
16.2%  in  Imaging  and  16.0%  in  Main  Radiology.  During  the 
post-implementation  study,  scheduling  activities  and  receiving 
activities  were  observed  separately.  Receiving  occupied  5.9%  of  the 
time  in  Imaging  and  5.8%  in  Main  Radiology;  scheduling  occupied  15.4% 
of  the  time  in  Imaging  and  16.0%  in  Main  Radiology.  If  these  two 
functions  are  summed  for  each  section,  Imaging  receptionists  spent 
21.3%  of  their  time  and  Main  Radiology  receptionists  spent  21.8%  of 
their  time  devoted  to  scheduling  and  receiving  patients. 
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TABLE  5 


SUMMARY  OF  BASELINE  WORK  SAMPLING  IN  SCHEDULING  AND  RECEPTION  AREAS  — 

EVALUATION  OF  TRIRAD 
NAVAL  HOSPITAL,  BETHESDA 


Imaging 

Main 

Activity  Category 

Obser¬ 

vations 

Percent 

of 

Total 

Obser¬ 

vations 

Percent 

of 

Total 

Receive  and/or  schedule 
patients 

33 

16.22 

34 

16.02 

Prepare  flash  cards 

— 

— 

5 

2.3 

Type  onto  daily  log 

2 

1.0 

3 

1.4 

Answer  in-person  inquiries 
from  patients 

— 

— 

15 

7.0 

Answer  telephone  inquiries 

20 

9.8 

8 

3.8 

Make  telephone  Inquiries 

7 

3.4 

4 

1.9 

Distribute  519s 

1 

0.5 

— 

— 

Refile  patient  cards 

6 

2.9 

— 

— 

Check  exam  schedule 

7 

3.4 

14 

6.6 

Inspect  519s 

2 

1.0 

— 

— 

Inspect  519s  against  worksheet 

— 

__ 

1 

0.4 

Talk  to  hospital  staff 

14 

6.9 

11 

5.2 

Request  files 

— 

— 

2 

0.9 

Prepare  file  folder 

2 

1.0 

1 

0.4 

Other  productive  work 

28 

13.7 

15 

7.0 

Other  (personal,  away  from  area) 

82 

40.2 

100 

47.1 

TOTAL 

204 

100.02 

213 

100.02 

TABLE  6 


SUM11ARY  OF  POST- IMPLEMENTATION  WORK  SAMPLING  IN  SCHEDULING  AND  RECEPTION  AREAS  - 

EVALUATION  OF  TRIRAD 
NAVAL  HOSPITAL,  BETHESDA 


Activity  Category 
At  Reception  Area 
Receive  Patients 

Answer  inpatient  inquiries  from  patients 
Instruct  patients  in  treatment  preparation 
Answer  in-person  inquiries  from  physicians 
Answer  telephone  inquiries  re:  test  results 
Answer  other  telephone  inquiries 
*  Make  telephone  inquiries 
Talk  to  hospital  staff 
Wait  for  terminal 

Clerical 

Check  exam  schedule 
Prepare  file  jackets 
File  reports 
Type 

Request  films 
Computer  Functions 

Register  patients 
Schedule/cancel  exams 
Log  in  patient  arrival/departure 
Retrieve  patient  studies/schedule/data 
Edit  file 
Record  studies 
Print  labels 

Retrieve  day  log/schedule 
Identify  film  location 
Print  report 
Review  report 
Merge  files 

Awav  From  Reception  Area 

Search  for  physician/technician 
Distribute  reports 
Locate/pick  up  patient  files 
Escort  patient 
Other  duties  away  from  area 

Other 

Meal  Break 
Personal 
Inactive 

TOTAL 


Imaging 

Main 

Percent 

of 

Observations 

Total 

Observations 

21 

5.9% 

31 

18 

5.1 

34 

1 

0.3 

- 

9 

2.5 

4 

6 

1.7 

1 

33 

9.3 

32 

22 

6.2 

4 

33 

9.3 

8 

1 

0.3 

- 

4 

1.1 

1 

25 

7.0 

50 

13 

3.7 

- 

5 

12 

3.4 

- 

55 

15.4 

86 

21 

5.9 

30 

16 

4.  5 

7 

- 

- 

- 

1 

0.3 

1 

5 

1.4 

26 

3 

0.8 

- 

- 

- 

1 

2 

0.6 

- 

- 

- 

- 

— 

- 

5 

1.4 

- 

- 

- 

_ 

- 

1 

— 

- 

1 

24 

6.7 

50 

25 

7.0 

53 

1 

0.3 

40 

_ 

- 

67 

356 

100.0% 

533 

^Percentages  may  not  add  to  100%  due  to  rounding. 


Percent 

of 

Total 


5.8% 

6.4 

0.8 

0.2 

6.0 

0.8 

1.5 


0.2 

9.4 


0.9 


16.0 

5.6 

1.3 

0.2 

4.9 

0.2 


0.2 

0.2 

9.i 


9.9 

7.5 

12.6 


100.0% 
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In  boch  sections  there  was  an  increase  In  answering  telephone 
Inquiries  —  9.8Z  to  11. OZ  in  Inaging  and  3.82  to  6.22  in  Main 
Radiology. 

The  proportion  of  tine  spent  in  preparing  file  jackets  increased 
fron  1.02  to  7.0Z  in  laaglng  and  froa  0.42  to  9.42  in  Main  Radiology. 
This  apparent  Increase  is  due  to  a  change  in  procedures.  Formerly, 
file  jackets  were  prepared  Sy  file  root*  personnel.  Since  bar-code 
labels  for  the  file  jacket  were  printed  at  the  reception  area  during 
the  post-implementation  period,  this  activity  shifted  with  regards  to 
the  location.  This  measured  increase  is  not  believed  to  be  a  real 
increase  but  merely  a  change  in  process. 

c.  Patient  Waiting  and  Examination  Tine 

1.  Data  Considerations 

The  time  from  patient  arrival,  through  examination,  until  patient 
departure  was  measured  in  the  baseline  and  post-implementation 
periods. 

During  the  baseline  period,  the  progress  of  actual  patients  was 
followed  by  data  collectors  in  order  to  obtain  arrival  and  departure 
times.  In  Urologic  Radiology  and  Special  Procedures,  these  times  were 
derived  from  limited  observations  and  from  interviews  with  Radiology 
staff  to  verify  that  these  observations  were  typical  of  normal 
operations. 

During  the  post-implementation  data  collection,  the  computer¬ 
generated  daily  log  report,  which  contains  patient  arrival  and  de¬ 
parture  times,  was  used  whenever  possible.  In  each  section,  a  small 
sample  of  patients  were  followed  in  order  to  verify  the  accuracy  of 
the  computer  log.  In  cases  where  discrepancies  were  identified, 
patient-waiting-time  data  were  collected  by  tracking  patients  directly 
in  a  manner  similar  to  that  used  for  baseline  data  collection. 

ii.  Results 

The  data  for  the  baseline  and  post-implementation  periods  are 
given  in  Table  7.  Patient  waiting  and  examination  times  increased  in 
five  sections  of  the  Department  and  decreased  in  three  sections.  The 
reasons  for  the  increases  appear  to  be  due  to:  (I)  longer  examination 
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TABLE  7 


COMPARISON  OF  BASELINE  AND  POST-IMPLEMENTATION 
PATIENT  WAITING  AND  EXAMINATION  TIME 
EVALUATION  OF  TRIRAD 
NAVAL  HOSPITAL,  BETHESDA 


Patient  Waiting 
Baseline 


and  Examination  Time _ 

Post- Implementation 


Department  Section 

Average 

Main  Radiology 

40 

min. 

GI  Radiology 

66 

min. 

Orthopedic  Radiology 

34 

min. 

Urologlc  Radiology 

120 

min.a 

Special  Procedures 

120 

min.a 

Nuclear  Medicine 

96 

min. 

Computerized  Tomography 

60 

min. 

Ultrasonography 

125 

min. 

Sta.  Dev. 

Average 

Sta.  Dev. 

30  min.a 

54  min. 

49  min. 

22  min. 

115  min. 

73  min. 

16  min. 

50  min. 

26  min. 

— 

71  min. 

40  min. 

— 

107  min.*5 

64  min. 

43  min. 

176  min. 

126  min. 

28  min. 

83  min. 

41  min. 

37  min. 

49  min. 

17  min. 

Derived  from  limited  observations  and  estimates  where  needed. 

^Small  sample  size,  however,  for  Special  Procedures  this  represents 
a  full  day's  volume. 


does  in  GI  Radiology  and  Nuclear  Medicine  due  Co  changes  in  patient 
mix;  (2)  increased  elapsed  time  to  obtain  prior  film  files  in  Main 
Radiology;  (3)  increased  processing  time  due  to  TRIRAD  data 
requirements  and  changes  in  clinic  procedures  in  Orthopedic  Radiology; 
and  (4)  a  change  in  scheduling  procedures,  independent  of  TRIRAO  in 
Computerized  Tomography. 

The  reasons  for  decreases  in  processing  times  appear  due  to: 

(1)  a  change  in  radiologist  and  procedures  in  Ultrasonography;  and 

(2)  a  possible  change  in  mix  of  examinations  and  non-representative 
mean  process  times  derived  from  limited  observations  during  the 
baseline  period  for  Urologic  Radiology  and  Special  Procedures. 

The  interpretation  of  these  changes  is  discussed  further  in 

Chapter  V. 

.  d.  Time  for  Film  File  Requests 

i.  Data  Considerations 

During  the  baseline  period,  requests  for  prior  films  were  not 
recognized  as  a  problem  nor  was  it  believed  that  TRIRAD  would  impact 
this  segment  of  the  process.  However,  during  the  post-implementation 
period  Increases  in  patient  process  time  were  noted  in  Main  Radiology. 
During  interviews  with  staff  to  identify  possible  reasons  for  this 
increase,  requests  for  film  files  were  believed  to  be  a  major  cause  of 
delays.  Accordingly,  data  were  collected  on  this  segment  of  the 

process. 

ii.  Results 

The  measured  data  are  tabulated  in  Table  8.  File  requests  were 
made  by  the  receptionists.  The  patient's  file-jacket  labels  were  sent 
to  the  film  library  on  a  small  elevator,  known  as  a  dumbwaiter.  The 
average  time  that  elapsed  between  placing  the  labels  (which  was 
understood  as  a  request  for  prior  films)  in  the  dumbwaiter  until  the 

requests  were  received  in  the  film  library  was  14  minutes.  The 

average  time  that  elapsed  before  a  film  or  note  indicating  "not  in 
file"  was  returned  to  reception  was  21  minutes.  The  average  combined 
time  in  reception  from  request  to  return  from  film  library  was 
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measured  as  36  minuCes  (approximately  the  sum  of  the  two  segments  of 
the  process). 

After  the  prior  file  jacket  was  received  In  reception  It  was 
brought  to  the  quality  control  desk.  Also,  when  the  examination  had 
been  completed,  the  requisition  for  the  examination  was  placed  on  the 
control  desk.  After  the  films  were  developed,  they  were  matched  with 
the  requisition  and  the  prior  film.  The  elapsed  time  from  the  placing 
of  the  requisition  on  the  control  desk  until  the  placing  of  prior 
films,  or  supplemental  note,  on  the  control  desk  was  measured  and 
found  to  average  30  minutes. 

e.  Radiologist  Time  Devoted  to  Interpreting  Films 

i.  Data  Considerations 

Reading  time  was  evaluated  in  all  eight  sections  of  the  Radiology 
Department.  During  the  baseline  studies,  reading  times  in  Urologic 
Radiology  and  Special  Procedures  were  derived  from  limited  observa¬ 
tions  and  interviews  with  the  radiologists  to  determine  whether  these 
observations  were  typical  of  normal  operations. 

ii.  Results 

A  comparison  of  the  results  of  the  baseline  and  post-implementa¬ 
tion  periods  is  shown  in  Table  9  and  Figures  12  and  13.  In  Main 
Radiology,  Nuclear  Medicine  and  Computerized  Tomography,  a  measured 
decrease  in  reading  time  was  noted  from  baseline  to  post-implementa¬ 
tion.  A  decrease  was  also  noted  in  Urologic  Radiology  and  Special 
Procedures  from  a  reading  time  which  was  derived  from  limited  obser¬ 
vations  in  the  baseline  period  to  a  measured  reading  time  during  the 
post-implementation  period.  Reading  time  remained  constant  in  Ortho¬ 
pedic  Radiology  and  Ultrasonography.  A  slight  increase  in  reading 
time  was  measured  in  GI  Radiology. 

Concurrent  with  the  implementation  of  TRIRAD,  substantial  changes 
in  Radiology  staffing  and  procedures  were  introduced.  In  Chapter  5, 
Discussion  of  Results,  the  interpretation  of  these  changes  is 
discussed  in  detail. 

In  most  sections,  a  consistent  method  of  reading  films  was 
followed  throughout  the  study  period.  However,  in  the  Main  Radiology 
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TABLE  9 


COMPARISON  OF  BASELINE  AND  POST-IMPLEMENTATION  READING  TIMES 

EVALUATION  OF  TRIRAD 
NAVAL  HOSPITAL,  BETHESDA 

_ Reading  Times _ 

_ Baseline _  Post-Implementation 


Department  Section 

Average 

Sta.  Dev. 

Average 

Sta.  Dev 

Main  Radiology 

6  min.a,k 

4  min. 

3  min.a 

3  min. 

GI  Radiology 

10  min. 

8  min. 

11  min. 

4  min. 

Orthopedic  Radiology 

5  min. 

3  min. 

5  min. 

3  min. 

Urologic  Radiology 

10  min. c 

— 

7  min. 

2  min. 

Special  Procedures 

15  min.c 

— 

10  min. 

5  min. 

Nuclear  Medicine 

5  min. 

3  min. 

5  min. 

3  min. 

Computerized  Tomography 

12  min. 

7  min. 

7  min. 

7  min. 

Ultrasonography 

6  min. 

6  min. 

7  min. 

4  min. 

Analysis  of  changes  In  reading  times  suggest  that  differences  may  be 
due,  in  part,  to  staffing  and  procedural  changes  and,  in  part,  to 
TRIRAD. 

^Baseline  figures  are  for  handwritten  reports  only. 

c 

Refer  to  Chapter  5,  Discussion  of  Results,  for  detail. 


FIGURE  13  DISTRIBUTION  OF  POST  IMPLEMENTATION  READING 
TIMES  FOR  REPORTS  IN  MAIN  RADIOLOGY  - 
EVALUATION  OF  TRIRAD,  NAVAL  HOSPITAL, 
BETHESDA 
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Wet  Reading  Room  seven  different  methods,  or  combinations  of  methods, 
were  employed  in  the  post-implementation  period.  The  times  associated 
with  each  method  varied  as  can  be  seen  in  Table  10.  The  data  are 
grouped  by  reporting  method  with  and  without  computer  assistance.  For 
40%  of  the  volume  observed  in  the  Wet  Reading  Room,  bar-coded 
reporting  was  used  for  part  or  all  of  the  report  preparation.  The 

*  reading  time  for  these  computer-assisted  reports  averaged  2.0  minutes. 
The  60%  of  the  reports,  that  were  prepared  without  computer 
assistance,  took  3.0  minutes  to  Interpret  and  report. 

A  comparison  of  these  two  values  with  and  without  computer 
*  assistance  appears  to  indicate  that  bar  coding  speeds  up  interpreting 

and  reporting  of  films  and  to  suggest  that  widespread  use  of  bar 
coding  might  result  in  an  overall  reduction  in  reading  time.  This 

*  comparison  is  not  valid  to  make  with  these  data  because  the  mix  of 
examinations  was  different  in  the  two  groups.  Bar  coding  was  used  to 
create  simpler  reports  of  less  complex  interpretations.  Dictation  was 
used  for  more  complex  reports  for  which  bar-coded  statements  were  not 
available  or  were  awkward  to  use.  Bar  coding  may  or  may  not  reduce 
reporting  time  but  it  is  not  possible  to  draw  definitive  conclusions 
with  the  confounding  variables  in  these  data. 

f.  Time  For  Report  Transcription 

1.  Data  Considerations 

The  time  spent  transcribing  reports  was  measured  in  the 

* 

post-implementation  period  but  not  in  the  baseline  period  for  two 
reasons:  (1)  transcription  was  done  in  another  department  (Patient 
Affairs)  and  authorization  was  not  obtained  to  collect  these  data;  and 

l 

(2)  much  of  the  transcription  was  provided  by  a  contract  typing 
service  whose  control  was  outside  of  the  hospital  and  the  scope  of 
this  study.  Though  transcription  labor  time  spent  per  report  could 
not  be  evaluated  quantitatively  during  the  baseline  period,  and  chough 
a  direct  comparison  of  change  could  not  be  made,  transcription  labor 
was  measured  during  the  post-implementation  study. 

*  The  elapsed  time  between  arrival  of  the  radiologists'  dictation 

until  transcription  was  complete  was  studied  during  both  periods. 
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ANALYSIS  OF  POST-IMPLEMENTATION  READING  TIMES  BY  REPORTING  METHOD 
IN  THE  MAIN  RADIOLOGY  WET  READING  ROOM 
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Percentages  may  not  add  to  100%  due  to  rounding. 


During  the  baseline  period  Che  elapsed  time  segment  In  transcription 
was  studied  because  this  part  of  the  process  was  reported  by  Radiology 
staff  to  contribute  to  delays  in  turnaround  time  for  reports.  During 
the  post-implementation  period  this  was  observed  to  no  longer  be  the 
case.  During  the  post-implementation  period  the  examination  date  and 
transcription  date  were  recorded  and  compared. 

li.  Results 

Data  on  post-implementation  transcription  times  are  shown  on 
Table  11,  which  contains  the  number  of  observations,  the  average  time 
and  the  standard  deviation  by  section.  The  table  also  contains  an 
estimate  of  the  number  of  reports  that  could  be  transcribed  per  hour. 
Average  transcription  times  (and  reports  per  hour)  ranged  from  a  low 
of  53  seconds  (60  per  hour)  in  Ultrasonography  and  1  minute,  8  seconds 
(60  per  hour)  in  Orthopedic  Radiology  to  a  high  of  4  minutes,  10 
seconds  (15  per  hour)  in  Computerized  Tomography  and  an  even  longer 
report  in  Special  Procedures.  In  Ultrasonography,  reports  were  typed 
from  handwritten  notes  according  to  a  "screen"  or  framework  of 
pre-def lned  statements  that  permitted  rapid  typing.  In  Orthopedic 
Radiology  reports  were  typed  from  handwritten  notes  and  the  reports 
tended  to  be  either  brief  or  used  pre-def ined  statement  codes,  thus 
explaining  the  short  transcription  time.  In  Computerized  Tomography 
the  reports  were  lengthy  and  complex.  Reports  for  Special  Procedures 
probably  required  more  transcription  time  than  other  reports;  however, 
this  is  difficult  to  evaluate  based  on  measured  data,  as  only  one  such 
report  was  observed.  (Though  an  actual  comparison  of  labor  time  for 
transcription  is  precluded  by  the  absence  of  baseline  data,  Radiology 
staff  believed  that  the  power  typing  capabilities  of  TRIRAD  had 
increased  the  efficiency  of  transcriptionists. ) 

Mean  elapsed  time  in  transcription  during  the  baseline  period 
varied  between  7  end  21  days  for  different  sections  of  Radiology, 
substantially  adding  to  the  delays  in  report  turnaround  time  ( data  for 
this  time  segment  by  section  are  contained  in  Appendix  B).  During 
post-implementaticn  observations  this  segment  was  found  to  average 
less  than  one  day.  (A  more  precise  value  was  not  possible  because  the 


POST-IMPLEMENTATION  TRANSCRIPTION  TIME 
EVALUATION  OF  TRIRAD 


Includes  23  dlrect-dict:ation-to-transcrlptionist  observations  (average  2  min.  56  sec.)  and  10 
dlctation-tape-to-transcriptionist  observations  (average  3  min.  18  sec.). 


delivery  of  dictation  types  to  transcription  could  not  be  reliably 
determined.)  The  elapsed  time  for  this  segment  has  decreased 
substantially  since  the  baseline  period  with  a  resultant  decrease  in 
report  turnaround  time. 

3.  Film  File  Availability 
*  i.  Data  Considerations 

Data  concerning  film-file  availability  were  collected  during 
observation  of  radiologists  in  the  reading  rooms.  It  was  believed 
that  the  Impact  of  improved  file  availability  would  be  greatest  in  the 
reading  rooms  since  the  largest  volume  of  prior  films  were  pulled 
expressly  for  comparisons  in  the  reading  rooms.  Also,  it  was  believed 
that  changes  in  file  availability  in  the  reading  rooms  could  have  a 
direct  impact  on  the  quality  of  patient  care.  For  each  reading 
observed,  the  presence  or  absence  of  prior  films  was  noted. 

ii.  Results 

Tables  12  and  13  show  the  results  concerning  file  availability  in 
the  baseline  and  post-implementation  periods,  respectively.  The  upper 
half  of  each  table  shows  the  number  of  prior  films  that  were  or  were 
not  present  in  the  reading  room;  "don’t  know"  was  indicated  by  the 
data  collector  when  it  could  not  be  determined  whether  a  prior  film 
was  available  or  not.  The  lover  half  of  the  table  shows,  for  prior 
films  in  the  reading  room,  whether  they  were  In  fact  used  in  the 
interpretation.  Note  that  in  GI  Radiology  it  was  more  difficult  to 
determine  whether  prior  films  were  used. 

Examination  of  the  results  of  the  total  of  all  sections  indicates 
that  during  the  baseline  period  41*  of  all  interpretations  had  a  prior 
file  available,  42%  did  not,  and  in  17%  of  the  cases  the  availability 
of  prior  films  could  not  be  determined.  By  comparison,  during  the 
post-implementation  period  32%  of  the  interpretations  had  a  prior  film 
available,  50%  did  not  and  for  18%  the  availability  of  prior  films 
could  not  be  determined.  A  comparison  of  the  total  volumes  shows  that 
9%  fewer  interpretations  during  the  post-implementation  period  had  a 
prior  film  available  for  comparison.  It  is  important  to  recognize,  in 
interpreting  these  results,  that  no  distinction  was  made  as  to  the 


ANALYSIS  OF  POST- IMPLEMENTATION  AVAILABILITY  OF  PRIOR  FILMS  DURINC  INTERPRETATION 
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In  Cl  Radiology,  ra.1  lologlats  usually  participate  In  taking  the  examination.  Before  the  official  reading  session, 

prior  files  may  be  reviewed.  The  numbers  reflected  here  Indicate  only  the  use  of  prior  films  In  the  observed  reading  section 
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type  of  patient  or  to  the  reason  why  a  prior  film  might  not  have  been 
present.  The  purpose  of  this  study  focused  on  the  availability  of 
prior  films  to  the  radiologist  at  the  time  of  the  official  reading. 
Thus  these  data  were  collected  in  the  reading  room.  It  was  not 
possible  to  assess  which  current  films  should  have  had  a  prior  film  in 
either  the  baseline  or  post- implementation  period.  Accordingly,  films 
for  first  time  patients  and  films  out  on  loan  were  both  recorded  as 
"not  available"  to  the  radiologist  during  the  official  reading.  This 
methodology  was  consistently  followed  in  both  study  periods. 

In  the  Radiology  Wet  Reading  Room,  the  percentage  of  available 
films  decreased  from  40%  in  the  baseline  period  to  14%;  observations 
were  not  made  in  the  Inpatient  Reading  Room  during  the  baseline  period 
so  a  comparison  is  not  possible;  in  GI  Radiology  a  decrease  from  50% 
to  30%  was  measured;  in  Orthopedic  Radiology  a  decrease  from  60%  to 
46%;  and  in  Computerized  Tomography  from  33%  in  the  baseline  down  to 
10%  during  post-implementation.  Improvements  were  measured  in  Nuclear 
Medicine  (from  40%  to  47%)  and  in  Ultrasonography  (from  0%  to  18%) . 

Comparison  of  the  number  of  available  prior  films  that  were,  in 
fact,  used  in  the  interpretation  shows  an  increase  from  70%  in  the 
baseline  period  to  75%  during  the  post-implementation  period.  A 
calculation  of  the  total  number  of  prior  films  used  divided  by  the 
total  number  of  observations  shows  that  for  29%  (73  of  252  observa¬ 
tions)  of  all  interpretations  a  prior  film  was  actually  used  during 
the  baseline  period  and  for  24%  (82  of  343  observations)  of  all 
interpretations  a  prior  film  was  actually  used  during  the  post- 
implementation  period,  or  a  net  reduction  of  5%  of  the  total  volume. 

C.  COSTS  OF  RADIOLOGY  SERVICES  AND  TRIRAD  EQUIPMENT 

Information  on  costs  of  Radiology  services  was  obtained  from 
budgeting  figures  contained  in  the  Uniform  Chart  of  Accounts  (UCA) 
reports  provided  by  the  Comptroller  of  NHB.  The  reader  should  be 
aware  that  the  budgeting  figures  should  be  used  as  a  general  reference 
only  and  for  overall  comparison  with  the  costs  of  TRIRAD  equipment  and 
operation. 
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Table  14  shows  the  cost  figures  for  both  FY81  (last  full  year 
prior  to  TRIRAD  installation)  and  FY82  (TRIRAD  was  installed  during 
the  second  quarter  of  FY82);  for  the  following  cost  elements: 

•  Direct  expenses  -  including  labor,  supplies,  utilities, 
etc. ; 

•  Indirect  expenses  -  reported  as  support  costs  by  UCA; 

•  Radiology  equipment  -  major  capital  equipment,  not 
including  TRIRAD; 

•  Maintenance  and  repair;  for  major  capital  equipment,  not 
including  TRIRAD;  and 

•  TRIRAD  operating  expenses  -  for  FY82  only,  includes  the 
operating  expenses,  attributable  to  TRIRAD,  for  the 
hospital's  main  computer  room. 

In  FY81  Radiology  services  cost  $5,677,107  and  for  FY82  costs 
were  $6,416,925.  (Note  carefully  the  extensive  footnotes  on  Table  14 
that  discuss  assumptions  and  methods  used  in  preparing  these  cost 

4 

figures.)  In  FY81  the  volume  of  examinations  totaled  84,574  and  the 
workload  totaled  493,462^  weighted  work  units.  In  FY81  the  cost  per 
examination  was  $67.13  and  the  cost  per  work  unit  was  $11.50.  In  FY82 

4 

the  volume  of  examinations  totaled  92,816  and  the  workload  totaled 
776,267^  weighted  work  units.  In  FY82  the  cost  per  examination  was 
$69.13  and  the  cost  per  work  unit  was  $8.27  (not  including  the  cost  of 
TRIRAD). 

The  changes  in  the  Radiology  costs  from  FY81  to  FY82  appear  to 
have  been  due  to  several  identifiable  factors  and  were  probably  due, 
in  part,  to  other  factors  that  have  not  been  identified.  The 
operating  cost  of  the  TRIRAD  mainframe  has  been  added.  The  two 


Note  that  the  apparent  increase  in  volume  between  FY81  and  FY82  may 
be  due,  entirely  or  in  part,  to  the  improved  capability  with  TRIRAD  to 
record  and  account  for  each  procedure  performed. 

^Note  that  the  apparent  increase  in  workload  as  recorded  in  weighted 
work  units  is  due  primarily  tc  the  changes  in  the  Nuclear  Medicine 
section  for  assigning  weights  to  each  examination. 
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TABLE  14 


ANNUAL  BUDGET  FOtf  RADIOLOGY  SERVICES®  IN  FY81  AND  FY82  — 
EVALUATION  OF  TRIRAD 
NAVAL  HOSPITAL,  BETHESDA 


Cost  Element 

Direct  expenses  (in¬ 
cludes  labot,  supplies, 
utilities,  etc.) 

Indirect  expenses 

Radiology  equipment** 

0 

Maintenance  and  repair 
TRIRAD  operating  expenses^ 


$5,677,107  $6,416,925 

Includes  Diagnostic  Radiology  and  Nuclear  Medicine  sections,  but  does  not 
include  Radiation  Therapy  since  this  section  does  not  use  TRIRAD. 

bFY81  figures  used  as  baseline  values  since  last  full  year  prior  to  TRIRAD 
installation.  FY82  figures  used  for  post-implementation  values  since  first  year 
of  TRIRAD  operation  although  TRIRAD  was  installed  during  second  quarter  of  FY82. 

Values  projected  by  multiplying  first  three  quarters  of  FY82  by  1.33  since 
values  for  fourth  quarter  are  not  yet  available. 

dRadiology  equipment  includes  major  capital  equipment,  but  not  TRIRAD  equipment. 
Value  of  equipment  determined  to  equal  $12  million,  allocated  over  7  years,  i.e. 
$12  million  *  7  years  *  $1,714,285.  Value  determined  from  Radiology  equipment 
records  and  interviews  with  staff.  Value  assumed  to  be  the  same  in  FY81  and  FY6 
since  no  major  equipment  purchases  were  made. 

0 

Maintenance  and  repair  on  major  capital  equipment  but  not  TRIRAD.  Based  upon 
maintenance  contracts  of  $420,000  and  estimate  of  in-house  maintenance  of 
$100,000.  Value  assumed  to  be  the  same  in  FY81  and  FY82  since  no  major  change 
in  equipment  to  be  maintained. 

^TRIRAD  operating  expenses  based  upon  third  quarter  of  FY 82  includes  the  opera¬ 
tion  of  the  main  frame  in  the  hospital's  main  computer  room.  Since  TRIRAD  was 
installed  from  second  through  fourth  quarters,  and  since  data  for  the  third 
quarter  are  believed  to  be  most  reliable,  FY82  values  were  determined  by  multi¬ 
plying  third  quarter  expenses  times  three,  i.e.,  $16,607  x  3  »  $49,821.  (NHB 
Comptroller  staff  felt  that  second  quarter  data  were  unreliable  due  to  failure 
to  capture  all  costs  incurred  during  the  start-up  period.) 
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transcriptionists  hired  to  work  within  Radiology  during  FY82  replaced 
the  transcriptionlst  from  Central  Transcription  and  the  outside  typing 
services  used  in  FY81;  on  the  Radiology  budget  transcription  was 
included  as  a  direct  expense  in  FY82  whereas  it  was  listed  as  an 
Indirect  expense  in  FY81.  It  appears  that  there  may  be  no  net  change 
in  total  cost  as  a  result  of  this  management  change.  Inflation  in 
salaries,  supplies,  utilities  and  other  expenses  account  for  a  portion 
of  the  change.  The  Increase  in  volume  of  examinations  requires  a 
corresponding  Increase  in  supplies,  including  direct  and  indirect 
expenses  for  these  supplies. 

The  list  of  TRIRAD  equipment  costs  for  FY82  is  contained  in  Table 
15  and  totals  $232,161.  TRIRAD  costs  include  hardware  lease,  hardware 
maintenance,  software  lease  and  maintenance  and  one-time  charges. 
Table  15  lists  each  component  of  the  TRIRAD  system  installed  at  NHB, 
the  number  of  units,  the  monthly  cost  per  item  and  the  cost 
contributed  by  each  component  during  FY82.  For  example,  the  four 
intelligent  KVDT's  were  leased  at  a  rate  of  $125  per  month  and  were 
Installed  for  8.77  months  during  FY82;  as  a  result,  the  hardware  cost 
for  these  terminals  is  found  to  be  $4,383  (4  units  x  $125  per  month  x 
8.77  months).  In  succeeding  years  the  annual  cost  figure  will  be 
calculated  for  12  months. 

The  one-time  charges  of  $110,650  include  such  items  as  training, 
installation  and  documentation.  These  one-time  charges,  as  the  term 
suggests,  have  only  occurred  during  the  first  year  of  the  contract  and 
will  not  be  incurred  again.  Thus  in  succeeding  years,  if  software  and 
hardware  costs  remain  constant,  the  annual  cost  figure  is  expected  to 
be  lower,  e.g.,  in  FY83  an  annual  figure  of  $176,000  has  been 
predicted.  Appendix  E  contains  details  of  TRIRAD  equipment  costs  for 
succeeding  years  through  March  1989  and  a  detailed  table  of  one-time 
charges. 

A  comparison  of  TRIRAD  equipment  and  operating  costs  with  FY82 
budget  figures  shows  that  TRIRAD  represents  4.4%  of  the  FY82  budget 
[($232,161  +  49,821)  ($6,416,925  x  100  -  4.4%)].  With  the  expected 
increases  in  the  Radiology  budget  during  each  succeeding  year  and  the 
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TABLE  15 


COSTS  OF  TRIRAD  SYSTEM  FOR  FY821  HARDWARE,  SOFTWARE  AMD  MAINTENANCE  — 

EVALUATION  OF  TRIRAD 
NAVAL  HOSPITAL,  BETHESDA 


Hardware  Lease 
8-Channel  Expander 
Bar  Code  Printer 
Bar  Code  Reader 
KVDT  Basic 
KVDT  Intelligent 
KAXIFILE  System 
Modem  Short  Haul. 

Modem  Short  Caul1 
Optical  Scanner 
Printer  150CPS 

Subtotal 

Hardware  Maintenance 
8-Channel  Expander 
Bar  Code  Printer 
Bar  Code  Reader 
KVDT  Basic 
KVDT  Intelligent 
MAXIFILE  System 
Modem  Short  Haul. 

Modem  Short  Haul1 
Optical  Scanner 
Printer  150CPS 

Subtotal 

Software  Lease  and  Maintenance 
Film  File  Management 
Patient  Record  File 
Scheduling  Feature 
Report  II-A  Module 
Report  II-B  Module 
Statistical  Package 
Teaching  File 

Subtotal 

Equipment  Lease  and 
Maintenance  Costs 

Software  Lease  and 
Maintenance  Costs 


Ho. 

Monthly  Coat 

Per  Item1 

FY822 

6 

$  70.00 

$  3,682 

7 

295.00 

18,103 

24 

45.00 

9,468 

17 

55.00 

8,197 

4 

125.00 

4,383 

1 

2,390.00 

20,952 

53 

10.00 

4,646 

1 

10.00 

37 

1 

525.00 

1,960 

6 

120.00 

6,312 

$77,740 

« 

6 

$  20.00 

S  1.105 

7 

130.00 

8,377 

24 

25.00 

5,523 

17 

25.00 

3,912 

4 

35.00 

1,289 

1 

1,400.00 

12,887 

3 

10.00 

4,879 

1 

10.00 

39 

1 

150.00 

588 

6 

50.00 

2,762 

$41,361 

1 

$  50.00 

$  438 

1 

50.00 

438 

1 

50.00 

438 

l 

75.00 

658, 

1 

75.00 

225? 

1 

50.00 

*> 

1 

75.00 

213 

92,410 
$119,101 
9  2,410 


One-Tine  Costa 


5 


9110,650 


TOTAL  COSTS 


$232,161 


jDollars  spent  or  committed  to  be  spent  by  TR1MIS  Program  Office  during  FY8 
FY82  costs  can  only  be  calculated  from  monthly  costs  if  date  of  installa¬ 
tion  for  each  itam  is  known.  Installation  of  equipment  and  software  coo- 
jponents  were  staggered  between  January  and  September,  1982 
^Delivery  delayed 
^Installed  FYS3 

For  breakdown  of  one-time  costs,  refer  to  Appendix  E 


67 


r 


lower  annual  cost  of  TRIRAD  equipment  (since  one-time  charges  will  not 
be  incurred) ,  It  is  anticipated  that  the  percentage  of  annual 
Radiology  costs  due  to  TRIRAD  equipment  will  be  lower  than  the  4.4% 
calculated  for  FY82. 

The  cost  of  Radiology  services  per  examination  was  shown  above  to 
be  $69.13  in  FY82.  If  the  costs  of  TRIRAD  equipment  are  added  to  the 
Radiology  budget  then  the  cost  per  examination  Increases  to  $71.64. 

D.  QUALITATIVE  FINDINGS 
1.  Questionnaire  Results 

a.  Characteristics  of  Respondent  Sample 

The  respondents  to  the  baseline  survey  included  65  Radiology 
Department  staff:  13  radiologists,  eight  residents,  37  technicians, 
and  seven  classified  as  other  (clerical  staff  and  physicists).  Thirty 
of  the  respondents  were  working  in  Diagnostic  Radiology,  15  were 
assigned  to  one  of  the  Imaging  sections  (3  in  Computerized  Tomography, 
10  in  Nuclear  Medicine,  and  2  in  Ultrasound),  and  15  were  assigned  to 
other  sections  (11  from  Radiation  Therapy,  2  from  Special  Procedures, 
and  2  from  Orthopedic  Radiology) .  Five  respondents  worked  or.  a 
rotating  basis  in  two  or  more  sections. 

Respondents  to  the  post-implementation  survey  included  53  staff: 
10  each  of  radiologists  and  residents,  22  technicians,  and  11  staff 
classified  as  other.  Of  these  20  were  assigned  to  Diagnostic 
Radiology,  23  were  assigned  to  one  of  the  imaging  sections,  and  seven 
were  assigned  to  Special  Procedures  (3),  Orthopedics  (2),  and  Urology 
(2).  Three  other  respondents  —  a  radiologist,  a  resident,  and  one 
technician  —  worked  on  a  rotating  basis  in  three  or  four  sections. 
Nearly  75%  of  the  respondents  had  been  on  the  staff  of  the  Radiology 
Department  prior  to  the  installation  of  the  TRIRAD  system. 

b.  Satisfaction  with  Radiology  Department  Operations 

i.  Introduction 

Both  the  baseline  and  post-implementation  surveys  solicited 
satisfaction  ratings  for  various  aspects  of  the  operation  of  the 
Radiology  Department  without  specific  reference  to  the  TRIRAD  system. 
Table  16  compares  weighted  mean  satisfaction  ratings  in  the  baseline 
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COMPARISON  OP  BASELINE  AND  POST-IMPLEMENTATION  SATISFACTION  OP  RADIOLOGY  DEPARTMENT  STAFF 

WITH  DEPARTMENT  OPERATIONS  — 

RESPONDENTS  TO  STAFF  SURVEYS  AT  NAVAL  HOSPITAL,  BETHESDA 
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aA  weighted  wean  satisfaction  response  was  obtained  by  assigning  a  value  of  +2  to  responses  of  "very  satisfied,"  *1  to 
"somewhat  satisfied,”  0  to  undecided,  -1  to  "eoawwhat  dlseatiaf led"  and  -2  to  "very  dissatisfied";  summing  these  values; 
and  dividing  the  sum  by  the  number  of  respondents  who  answered  the  question.  Thus  a  positive  result  Indicates  a  mean  level 
of  satisfaction  overall  and  a  negative  result  Indicates  an  overall  mean  level  of  dissatisfaction. 


COMPARISON  OP  BASELINE  AND  POST-IMPLEMENTATION  SATISFACTION  OF  RADIOI.OCY  DEPARTMENT  STAFF 

WITH  DEPARTMENT  OPERATIONS 

RESPONDENTS  TO  STAFF  SURVEYS  AT  NAVAL  HOSPITAL,  BETHESDA 


aA  weighted  mean  satisfaction  response  was  obtained  by  assigning  a  value  of  +2  to  responses  of  "very  satisfied,  *1  to 
"somewhat  satisfied,"  0  to  undecided,  -l  to  "somewhat  dissatisfied"  and  -2  to  "very  dissatisfied";  suomlng  these  values; 
and  dividing  the  sum  by  the  number'  of  respondenta  who  answered  the  question.  Thus  a  positive  result  Indicates  a  mean  level 
of  satisfaction  overall  and  a  negative  result  indicates  an  overall  mean  level  of  dissatisfaction. 


and  post-implemenCaCion  surveys.  Because  of  process  changes  brought 
about  by  TRIRAD,  the  relevant  aspects  of  operations  for  each  area  for 
the  baseline  were  not  identical  to  those  for  the  post-implementation 
operation.  Therefore,  specific  aspects  relating  to  areas  of  service 
effectiveness  have  been  grouped  into  categories  (e.g.,  ease  of 
obtaining  information,  completeness  of  Information).  Within  each 
category,  the  survey  items  for  baseline  service  aspects  are  listed 
first,  followed  by  the  relevant,  parallel  aspects  for  the  post-imple¬ 
mentation  process.  A  similar  table  (Table  17)  gives  the  distribution 
of  responses  of  radiologists  and  residents  into  the  different  satis¬ 
faction/dissatisfaction  categories  on  the  questionnaire. 

ii.  Timely  Availability  of  Films,  Records,  and  Reports 
As  shown  in  Table  16,  baseline  responses  of  radiologists  and 
residents  regarding  timely  availability  of  films,  records,  and  test 
results  indicated  they  were  somewhat  dissatisfied.  For  radiologists, 
the  lowest  mean  rating  was  for  test  results  for  routine  cases. 
Overall,  staff  radiologists  appeared  to  be  more  dissatisfied  with 
these  aspects  of  department  operations  than  were  residents.  For 
residents,  the  availability  of  historical  records  was  the  least 
satisfactory  aspect.  Overall,  both  technicians  and  other  staff 
appeared  to  be  essentially  neutral. 

In  the  post-implementation  period,  X-ray  films  were  still  filed, 
but  the  filed  copies  of  reports  were  no  longer  the  only  record, 
because  thev  were  also  stored  by  the  TRIRAD  system.  As  noted  in  Table 
16,  satisfaction  ratings  for  availability  of  reports/records  seem  to 
have  Increased  from  a  level  of  somewhat  dissatisfied  to  neutral.  The 
ratings  for  timely  availability  of  films  in  the  post-implementation 
survey  were  essentially  neutral  for  current  films,  but  indicating  some 
dissatisfaction  for  historical  films.  (The  data  base  of  stored 


reports  and  film  file  records  should  continue  to  increase  the  longer 
the  TRIRAD  system  is  in  use.)  Technicians  and  other  staff  appeared 
overall  to  be  somewhat  more  satisfied  with  these  aspects  of  operations 
in  the  post-implementation  survey. 
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The  Increasing  satisfaction  of  radiologists  and  residents  with 
availability  of  films  and  reports /records  is  also  apparent  in  the 
distribution  of  responses,  (Table  17).  In  the  baseline  survey,  from 
66%  to  922  of  all  radiologists  surveyed  expressed  some  dissatisfaction 
with  these  aspects  (including  those  who  were  somewhat  and  very 
dissatisfied).  No  respondents  indicated  they  were  "very  dissatisfied" 
with  the  availability  of  current  records  and  films  in  the  post¬ 
implementation  survey  and  402  of  radiologists  and  42%  of  residents 
registered  some  dissatisfaction  with  availability  of  historical  films. 

ill.  Ease  of  Obtaining  Information 

Baseline  satisfaction  with  the  ease  of  obtaining  records  and  test 
results  indicates  that  radiologists  and  residents  were  quite  consis¬ 
tently  "somewhat  dissatisfied"  (Table  16) .  Technicians  and  other 
staff,  however,  tended  to  be  neutral  or  somewhat  satisfied. 

In  the  post-implementation  survey.  Chough  all  respondents 
appeared  to  be  only  somewhat  more  satisfied  with  the  ease  of  obtaining 
films,  they  appeared  to  be  substantially  more  satisfied  with  the  ease 
of  obtaining  reports.  As  noted  in  Table  17,  only  20%  of  radiologists 
and  a  similar  percentage  of  residents  were  still  dissatisfied  (versus 
70%  of  radiologists  and  about  50%  of  residents  in  the  baseline 
survey).  A  comparison  of  survey  responses  for  technicians  and  other 
staff  likewise  Indicates  a  shift  from  overall  dissatisfaction  to  a 
level  of  "somewhat  satisfied"  (Table  16). 

The  post-implementation  satisfaction  expressed  by  both  groups  of 
physician  respondents  with  access  to  terminals  was  lower  than  that 
indicated  by  technicians  and  other  staff;  this  difference  is  believed 
to  have  resulted  from  the  fact  that  radiologists  and  residents  needed 
Intelligent  CRT  terminals  (these  intelligent  terminals  are  needed  for 
editing  and  approving  reports).  .There  were  only  four  such  terminals  i 
the  entire  Department,  and  all  transcription  had  to  be  done  at  these 
same  terminals  as  well.  Other  staff,  however,  had  access  to  24 
terminals,  all  capable  of  performing  the  other  necessary  computer 
functions. 
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iv.  Completeness  of  Information 

Baseline  responses  regarding  completeness  of  patient  records  and 
department  files  Indicate  a  consistent  level  of  dissatisfaction  for 
all  staff  types  (Table  16).  The  post-implementation  data  reveal  a 
substantial  shift  toward  greater  satisfaction  with  the  completeness  of 
the  patient  exam  history  (tests  conducted,  test  results),  which  was 
stored  by  TRIRAD  in  addition  to  the  paper  record  on  the  file  film 
jacket.  Whereas  no  physician  respondents  to  the  baseline  survey 
indicated  satisfaction  with  the  completeness  of  the  exam  history,  90% 
of  radiologists  and  60%  of  residents  did  so  in  the  post-implementation 
survey  (Table  17) .  Overall  satisfaction  of  technicians  increased  in  a 
similar  manner,  while  that  of  other  staff  remained  essentially 
unchanged  (Table  16) . 

The  post-implementation  results  for  completeness  of  patient  files 
indicate  more  satisfaction  than  in  the  baseline  survey  for  both 
physician  groups,  though  overall,  radiologists  were  still  somewhat 
dissatisfied  and  residents  dissatisfied.  Technicians  and  other  staff, 
who  were  overall  more  satisfied  with  file  completeness  in  the  baseline 
survey,  likewise  indicated  a  higher  level  of  satisfaction  than 
physicians  in  their  post-implementation  responses,  though  they  shifted 
from  slight  dissatisfaction  to  an  essentially  neutral  position. 

v.  Accuracy  of  Information 

The  baseline  survey  asked  for  satisfaction  ratings  for  the 
accuracy  of  information  on  the  paper  patient  record.  Radiologists 
overall  were  the  least  satisfied  (Table  16);  as  noted  in  Table  17, 
only  8%  of  those  surveyed  were  satisfied  with  accuracy  of  information. 
Residents,  on  the  other  hand,  were  largely  neutral. 

The  post-implementation  responses  of  these  two  groups  clearly 
show  greater  confidence  in  the  accuracy  of  the  information  when  stored 
on  the  computer  (Table  17),  with  80%  of  the  radiologists  and  70%  of 
the  residents  surveyed  reporting  that  they  were  "somewhat"  or  "very 
satisfied." 
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c. 


Overall  Satisfaction  with  TRIRAD 


The  post-implementation  survey  asked  Radiology  Department  staff 
to  rate  their  overall  satisfaction  with  TRIRAD.  Of  the  49  survey 
respondents  who  answered  the  question,  six  were  very  satisfied,  23 
were  somewhat  satisfied,  17  were  neutral,  three  were  somewhat 
dissatisfied,  and  none  were  very  dissatisfied.  The  distribution  of 


responses  by  staff  type  was 

as  follows: 

Very 

Somewhat 

Somewhat 

Satisfied 

Satisfied 

Neutral 

Dissatisf  li 

Radiologists 

2 

3 

4 

1 

Residents 

0 

7 

3 

0 

Technicians 

1 

10 

7 

2 

Other  Staff 

3 

3 

3 

0 

d.  Benefits 

of  TRIRAD 

1.  Expectations 

The  baseline  survey  solicited  opinions  regarding  the  likely 
benefits  of  a  radiology  management  system  in  the  department.  The 
results  are  given  in  Tables  18  and  19  for  two  categories  of  benefits: 
those  affecting  primarily  the  efficiency  and/or  economy  of  operations 
and  those  affecting  the  quality  of  services  provided.  Results  are 
shown  by  staff  type  and  for  the  63  survey  respondents  in  total. 

As  shown  in  Table  18,  survey  respondents  expected  some  beneficial 
effects  on  efficiency /economy  in  almost  every  impact  area.  The 
notable  •‘•seep  t  ion  is  reduction  in  the  staffing  requirements. 
Overall,  radiologists  appeared  to  have  higher  expectations  regarding 
beneficial  Impacts  than  the  other  staff  surveyed.  Those  impacts  rated 
as  most  likely  to  occur  included  making  the  patient's  current  record 
more  easily  accessible  and  available  more  quickly,  improving  the 
scheduling  of  patients,  and  reducing  the  number  of  telephone 
inquiries.  Respondents  overall  were  unconvinced  that  a  s/stem  would 
decrease  the  cost  of  services. 

The  results  concerning  expectations  for  possible  impacts  relating 
to  the  quality  of  service  are  shown  in  Table  19.  Respondents  appeared 
to  expect  improvements  in  file  maintenance,  record  completeness,  and 


TABLE  18 


EXPECTATIONS  OF  RADIOLOGY  DEPARTMENT  STAFF  OF  DIFFERENT  STAFF  TYPES  REGARDING  THE 
BENEFITS  OF  A  COMPUTERIZED  RADIOLOGY  MANAGEMENT  SYSTEM  FOR  EFFICIENCY  AND  ECONOMY  — 
BASELINE  STAFF  SURVEY  AT  NAVAL  HOSPITAL,  BETHESDA 


Impact 


Weighted  Mean  Response 


a,b 


Radio¬ 

logist 

Resi- 

dent 

Techni¬ 

cian 

Other 

Total 

Reduce  time  to  produce  a 
written  report 

1.6 

1.4 

1.7 

1.6 

0.9 

Make  patient  current  records 
more  easily  accessible 

1.5 

1.0 

0.9 

1.9 

1.2 

Make  patient  historical  rec¬ 
ords  more  easily  accessible 

1.5 

0.8 

0.7 

1.6 

1.0 

Make  patient  current  records 
available  more  quickly 

1.4 

0.9 

0.9 

1.6 

1.1 

Make  patient  historical  rec¬ 
ords  available  more  quickly 

1.3 

0.4 

0.8 

1.6 

1.0 

Make  Radiology  function 
more  smoothly 

0.8 

0.5 

0.6 

1.3 

0.7 

Decrease  the  time  spent  looking 
for  records/test  results 

1.7 

1.3 

1.2 

1.6 

1.3 

Improve  scheduling  of  patients 

1.2 

0.9 

1.0 

1.3 

1.1 

Decrease  the  cost  of  radiology 
services 

-0.2 

0 

0.2 

1.0 

0.1 

Improve  the  way  things  are  done 

1.2 

0.6 

0.5 

1.4 

0.7 

Reduce  the  number  of  people 
needed  to  work 

-0.2 

-0.6 

-1.1 

-0.4 

-0.8 

Reduce  the  number  of  telephone 
inquiries  for  test  results 

1.5 

0.9 

0.9 

1.7 

1.1 

A  weighted  mean  response  was  obtained  by  assigning  values  of  -*-2,  +1,  0,  -1,  and  -2 
to  the  five  possible  answers  ranging  from  "definitely  will"  to  "definitely  not"  and 
by  dividing  the  sum  of  these  by  the  number  of  responses.  Thus  a  positive  mean  value 
indicates  that  overall  a  computerized  radiology  management  system  is  expected  to 
have  the  stated  impact. 

^Respondent  base  included  13  radiologists,  eight  residents,  37  technicians,  and 
seven  other  staff;  mean  is  based  on  the  number  of  actual  responses  to  each 
question  (i.e.,  excluding  no-answer  responses). 
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TABLE  19 


EXPECTATIONS  OF  RADIOLOGY  DEPARTMENT  STAFF  OF  DIFFERENT  STAFF 
TYPES  REGARDING  THE  BENEFITS  OF  COMPUTERIZED  RADIOLOGY 
MANAGEMENT  SYSTEMS  FOR  THE  QUALITY  OF  SERVICE- 
BASELINE  STAFF  SURVEY  AT  NAVAL  HOSPITAL,  BETHESDA 


Impact 


Weighted  Mean  Response 


a,b 


• 

Radio¬ 

logist 

Reduce  the  number  of 
lost  files 

1.4 

Result  in  more  complete 
patient  records 

1.5 

Reduce  the  number  of  errors 
in  patient  records 

0.9 

Allow  better  patient  care 

0.9 

Make  reports  easier  to  read 

1.5 

Resi¬ 

dent 

Techni¬ 

cian 

Other 

Total 

0.9 

0.8 

1.7 

1.0 

0.8 

0.9 

1.9 

1.1 

0 

0.4 

1.3 

0.5 

0.6 

0.3 

1.3 

0.6 

0.9 

0.9 

1.7 

1.1 

A  weighted  mean  response  was  obtained  by  assigning  values  of  +2,  +1,  0,  -1, 
and  -2  to  the  five  possible  answers  ranging  from  "definitely  will"  to  "defi¬ 
nitely  not"  and  by  dividing  the  sum  of  these  by  the  number  of  responses.  Thus 
a  positive  mean  value  Indicates  that  overall  a  computerized  radiology  management 
system  is  expected  to  have  the  stated  impact. 

^Respondent  base  included  13  radiologists,  eight  residents,  37  technicians,  and 
seven  other  staff;  mean  is  based  on  the  number  of  actual  responses  to  each 
question  (l.e.,  excluding  no-answer  responses). 


report  legibility  vlth  greater  certainty  than  Improvements  In  record 
accuracy  and  patient  care.  Again,  radiologists  were  more  certain  that 
benefits  would  be  realized. 

The  results  for  the  same  series  of  questions  regarding  expected 
benefits  are  given  in  Tables  20  and  21,  which  show  the  responses 
according  to  level  of  experience  or  familiarity  with  computerized 
radiology  management  systems. 

Responses  to  the  survey  question  of  familiarity  are  shown  in 
Table  22.  As  noted,  a  small  number  of  respondents  (all  of  whom  were 
technicians)  reported  prior  experience  in  actually  using  a  system,  and 
a  large  number  (30-402  of  the  respondent  sample)  reported  no  prior 
familiarity  or  experience.  For  use  in  the  analysis  of  expected 
benefits  in  Tables  20  and  21,  responses  have  been  pooled  to  Include 
both  experience  with  TRIRAD  and  experience  with  any  other  radiology 
management  system  and  respondents  have  been  counted  only  once,  within 
the  highest  experience  category  they  Indicated. 

Survey  respondents  with  the  least  prior  familiarity/experience 
had  the  lowest  expectations  regarding  benefits,  and  those  with  some 
direct  experience  (having  used  a  system)  appeared  to  be  somewhat  more 
cautious  than  those  who  had  only  read  or  heard  about  computerized 
radiology  management  systems  or  seen  them  demonstrated. 

li.  Realized  Benefits 

The  results  of  the  post-implementation  survey  concerning  staff 
perceptions  of  realized  benefits  to  efficiency/economy  are  given  in 
Table  23.  A  large  number  of  the  respondents  reported  beneficial 
changes  for  most  impacts  affecting  efficiency/economy.  Individual 
benefits  reported  by  50Z  or  more  of  the  respondent  base  (27  of  53) 
included: 

•  reduced  time  to  produce  a  written  report; 

•  more  timely  availability  of  patient  historical  records; 

e  decreased  time  spent  looking  for  records /test  results;  and 

•  improved  scheduling. 

Two  areas  of  ef f iciency/economy  were  viewed  as  not  having  been 
Improved  by  TRIRAD.  Respondents  felt  that  TRIRAD  had  not  reduced 
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TABLE  20 


EXPECTATIONS  OF  RADIOLOGY  DEPARTMENT  STAFF  WITH  DIFFERENT  LEVELS  OF  PRIOR 
SYSTEM  EXPERIENCE  OR  FAMILIARITY  REGARDING  THE  BENEFITS  OF  A  COMPUTERIZED 
RADIOLOGY  MANAGEMENT  SYSTEM  FOR  EFFICIENCY  AND  ECONOMY  OF  OPERATIONS— 
BASELINE  STAFF  SURVEY  AT  NAVAL  HOSPITAL,  BETHESDA 


Weighted  Mean  Respon 

a,b 

se 

0 

Impact 

None 

Read/ 

Heard 

Only 

Saw 

Demon¬ 

strated 

Saw 

Used 

Used 

Reduce  time  to  produce  a 
written  report 

0.3 

1.4 

1.8 

1.3 

0.5 

i 

f 

Make  patient  current  records 
more  easily  accessible 

0.7 

1.4 

1.5 

1.6 

1.1 

Make  patient  historical  rec¬ 
ords  more  easily  accessible 

0.6 

1.0 

1.3 

1.6 

0.9 

V 

Make  patient  current  records 
available  more  quickly 

0.5 

1.4 

1.7 

1.6 

0.9 

Make  patient  historical  rec¬ 
ords  available  more  quickly 

0.5 

1.1 

1.5 

1.4 

0.9 

Make  Radiology  function 
more  smoothly 

0.8 

0.7 

0.8 

0.6 

0.8 

Decrease  time  spent  looking 
for  records/test  results 

1.3 

1.5 

1.5 

1.6 

1.0 

Improve  scheduling  of  patients 

0.8 

1.3 

1.0 

1.0 

1.0 

Decrease  the  cost  of 
radiology  services 

0.3 

0.3 

0.6 

-0.9 

-0.1 

Improve  the  way  things  are  done 

0.7 

0.7 

1.0 

1.1 

0.5 

• 

Reduce  the  number  of  people 
needed  to  work 

-0.5 

-1.0 

-1.0 

-0.7 

-0.7 

1 

Reduce  the  number  of  telephone 
inquiries  for  test  results 

0.8 

1.2 

1.7 

1.6 

1.1 

A  weighted  mean  response  was  obtained  by  assigning  values  of  +2,  +1,  0,  -1,  and 
-2  to  the  five  possible  answers  ranging  from  "definitely  will"  to  "definitely  not" 
and  by  dividing  the  sum  of  these  by  the  number  of  responses.  Thus  a  positive  mean 
value  indicates  that  overall  a  computerized  radiology  management  system  is  expected 
to  have  the  stated  impact. 

^Responses  have  been  pooled  to  Include  experience  with  TRIRAD  or  other  systems; 
each  respondent  is  counted  only  once,  within  the  highest  level  of  experience. 

See  Table  22. 
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TABLE  21 


EXPECTATIONS  OF  RADIOLOGY  DEPARTMENT  STAFF  WITH  DIFFERENT  LEVELS  OF  SYSTEMS 
FAMILIARITY /EXPERIENCE  REGARDING  THE  BENEFITS  OF  A  COMPUTERIZED 
RADIOLOGY  MANAGEMENT  SYSTEM  FOR  THE  QUALITY  OF  SERVICE  — 
BASELINE  STAFF  SURVEY  AT  NAVAL  HOSPITAL,  BETHESDA 


&  b 

Weighted  Mean  Response  * 


Impact 

None 

Read/ 

Heard 

Only 

Saw 

Demon¬ 

strated 

Saw 

Used 

Used 

« 

* 

Reduce  the  number  of  lost 
files 

0.8 

1.5 

1.3 

1.0 

0.1 

Result  in  more  complete 
patient  records 

0.9 

1.2 

1.5 

1.6 

0.6 

f 

Reduce  the  number  of  errors 
in  patient  records 

0.4 

0.8 

0.8 

0.4 

0.5 

Allow  better  patient  care 

0.4 

0.9 

1.2 

0.3 

0 

Make  reports  easier  to  read 

0.8 

1.2 

1.7 

1.6 

1.1 

£ 

A  weighted  mean  response  was  obtained  by  assigning  values  of  +2,  -*1,  0,  -1,  and 
-2  to  the  five  possible  answers  ranging  from  "definitely  will"  to  "definitely  not" 
and  by  dividing  the  sum  of  these  by  the  number  of  responses.  Thus  a  positive 
mean  value  Indicates  that  overall  a  computerized  radiology  management  system  is 
expected  to  have  the  stated  impact. 

^Responses  have  been  pooled  to  include  experience  with  TRIRAD  or  other  systems; 
each  respondent  is  counted  onlv  once,  within  the  highest  level  of  experience. 

See  Table  22. 
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TABLE  22 


LEVEL  OF  EXPERIENCE/FAMILIARITY  WITH  TRIRAD  OR  OTHER  COMPUTERIZED 
RADIOLOGY  MANAGEMENT  SYSTEMS  — 

RESPONDENTS  TO  BASELINE  STAFF  SURVEY  AT  NAVAL  HOSPITAL,  BETHESDA 


Number  of 

£ 

Respondents 

Staff  Type: 

Radiologist 

Resident 

Technician 

Other 

Level  of  Familiarity 

* 

1 

No  familiarity 

TRIRAD 

3 

5 

16 

4 

Other  System 

4 

4 

11 

2 

% 

Read /heard  about  but 
not  seen  or  used 

TRIRAD 

4 

3 

13 

2 

Other  System 

1 

1 

10 

0 

Seen  demonstrated 

TRIRAD 

3 

0 

1 

2 

Other  System 

3 

0 

2 

0 

Seen  used 

TRIRAD 

2 

0 

1 

1 

Other  System 

3 

1 

3 

0 

♦ 

Used/tried 

TRIRAD 

0 

0 

0 

0 

Other  System 

0 

0 

8 

0 

\ 

• 

Sk 

2^ 

Total  responses  add  to  more  than  the  respondent  base  of  65  because  many 
respondents  noted  several  answers. 


Total 


28 

21 


22 

12 


6 

5 


4 

7 


0 

8 
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personnel  requlremencs  (36  reporcing  no  benefit  and  eight  reporting  a 
benefit).  A  large  number  of  respondents  gave  a  neutral  response 
regarding  any  effect  on  the  cost  of  radiology  services;  of  those  who 
were  not  neutral,  however.  13  reported  no  benefit  and  eight  reported  a 
benefit. 

A  comparison  of  the  responses  regarding  expectations  from  the 
baseline  survey  (Table  18)  with  the  responses  in  Table  23  suggests 
that  expectations  that  costs  of  services  and  personnel  requirements 
would  not  be  reduced  were  consistent  with  reported  experience  with 

•  TRIRAD.  Other  expectations  regarding  improvements  in  accessibility  of 
records  (both  ease  and  speed  of  obtaining  records)  and  time  spent 
looking  for  record/test  results  also  appear  to  have  been  met  for  many 
staff. 

r 

Responses  regarding  realized  benefits  to  the  quality  of  services 
are  given  in  Table  24.  For  most  impacts,  a  large  number  of  respond¬ 
ents  reported  a  benefit.  Benefits  reported  by  more  than  507  of 
respondents  Included: 

•  increased  completeness  of  patient  records; 
e  improved  patient  care;  and 
e  improved  ease  of  reading  reports. 

Responses  regarding  changes  in  the  number  of  lost  files  and  errors  in 
patient  records  were  mixed.  When  the  responses  regarding  realized 

•  benefits  to  the  quality  of  service  are  compared  with  expectations  as 
expressed  in  the  baseline  survey  (Table  21),  it  appears  that  the 
expectations  regarding  improved  record  completeness  and  ease  of 

•  reading  reports  were  met.  Staff  in  the  baseline  survey  were  somewhat 
less  confident  that  the  number  of  errors  in  patient  records  would 
decline,  and  this  appears  to  have  been  confirmed  by  the  somewhat  mixed 
responses  concerning  this  benefit  in  the  post-implementation  survey. 
The  one  area  in  which  there  was  an  obvious  discrepancy  between 
expectations  and  reported  experience  is  the  number  of  lost  files. 
Baseline  responses  indicate  that  overall  staff  of  all  types  were 
expecting  a  decrease  in  the  number  of  lost  files,  with  radiologists 
the  most  convinced.  The  post-implementation  data  in  Table  24  reveal 
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that  a  large  number  (20  or  382  of  the  53)  of  respondents  gave  a 
neutral  response;  the  remainder  were  evenly  divided  as  to  whether  this 
Impact  had  been  realized  or  not. 

Hi.  Effect  of  TRIRAD  on  the  Work  of  Department  Staff 

The  post-implementation  survey  asked  respondents  to  agree  or 
disagree  with  a  series  of  questions  concerning  the  effect  of  TRIRAD  on 
their  work.  Results  are  shown  in  Table  25  for  the  physician 
respondents  in  the  same  (radiologists  and  residents),  for  technicians 
and  other  staff,  and  for  the  respondent  base  as  a  whole.  On  the 
questionnaire,  positive  and  negative  statements  concerning  TRIRAD  were 
intermingled.  In  the  table  they  have  been  reordered  so  that  negative 
statements  appear  together  at  the  end  of  the  list. 

Of  the  positive  statements,  502  or  more  of  the  respondents  (27  of 
53)  agreed  with  the  following: 

•  it  was  easy  to  learn  to  use  the  system; 

•  it  has  made  my  work  more  efficient; 

•  it  has  made  recordkeeping  easier  for  me; 

•  the  benefits  have  outweighed  the  disadvantages;  and 

•  I  expect  the  benefits  to  increase  in  the  future. 

The  maximum  number  of  respondents  expressing  disagreement  with  a 
positive  statement  was  16  (30%)  for  the  statement  concerning  the 

challenge  of  work. 

Results  for  the  four  negative  statements  indicate  very  few 
respondents  felt  that  the  system  had  had  the  stated  effect.  A  large 
number  (23)  of  respondents  gave  a  neutral  response  regarding  loss  of 
flexibility  in  their  work.  For  the  three  other  statements,  50%  or 
more  of  the  respondents  disagreed  that  such  an  effect  had  occurred: 

•  I  do  not  trust  the  information  I  obtain; 

•  I  personally  use  it  as  little  as  possible;  and 

•  some  decision-making  has  been  taken  away  from  me. 

Of  the  physician  respondents,  87%  disagreed  with  or  gave  a  neutral 
response  to  the  first  statement,  and  94%  disagreed  with  or  gave  a 
neutral  response  to  the  second. 
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2.  Interviews  with  Attending  Physicians 

Ten  attending  physicians  were  interviewed,  in  both  the  baseline 
and  post-implementation  periods,  regarding  their  satisfaction  with 
those  aspects  of  Radiology  services  that  might  be  affected,  directly 
or  indirectly,  by  TRIRAD.  All  physicians  were  frequent  users  of 
Radiology  services,  depended  upon  Radiology  for  diagnostic  information 
and  expressed  strong  interest  in  the  quality  of  services  being 
delivered.  The  physicians  were  from  varied  medical  and  surgical 
specialties,  anesthesiology  and  emergency  medicine.  The  J  •’sues 
addressed  were  timeliness  of  reports,  access  to  current  and  :j.ot 
films,  accuracy  of  reports,  access  to  Radiology  consultations,  t  of 
ordering  routine  and  STAT  examinations,  the  presence  of  reports  the 
inpatient  and  outpatient  medical  records,  and  the  features  of  D 
that  were  most  useful  to  physicians  outside  of  Radiology. 

a.  Timeliness  and  Filing  of  Radiology  Reports 

During  the  baseline  period,  all  physicians  were  very  dissatisfied 
with  report  turnaround  time.  All  physicians  interviewed  agreed  that 
Radiology  reports  were  more  timely  following  the  implementation  of 
TRIRAD.  Despite  improvements,  however,  many  reports  for  inpatients 
were  still  not  back  in  the  chart  by  the  time  the  patient  was 
discharged.  The  physicians  interviewed  considered  this  situation 
unacceptable. 

In  responding  to  questions  about  the  timeliness  and  filing  of 
interpretation  reports  for  outpatients,  most  physicians  agreed  that 
there  have  been  noticeable  improvements  in  both  areas  since  TRIRAD  was 
implemented.  They  believed  that  official  Radiology  reports  for 
examinations  ordered  during  the  previous  encounters  were  available 
more  often  at  the  next  encounter.  However,  they  all  agreed  that 
further  improvements  were  desirable. 

Several  physicians  described  their  own  independent  systems  for 
obtaining  and  recording  timely  Radiology  information  for  outpatients. 
These  systems  were  in  use  in  the  baseline  period  and  have  continued  to 
be  used  during  post-implementation.  One  physician  described  a  process 
he  had  developed  whereby  the  outpatient  mails  a  postcard  to  him  the 
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day  of  Che  examination  and  this  serves  as  a  reminder  to  Che  physician 
Co  go  to  Radiology  to  view  the  film.  Another  physician  reported  that 
he  obtains  Radiology  results  on  outpatients  by  going  to  the  Radiology 
Department  and  reading  the  film  himself  and/or  discussing  the  film 
with  a  radiologist.  (Though  this  practice  is  quite  usual  for 
inpatients,  these  physicians  had  adopted  it  for  outpatients  as  well.) 
Both  physicians  summarized  the  findings  themselves  in  the  notes  of  the 
patient  encounter. 

Several  physicians  described  specific  cases  where  patient  care 
could  have  been  adversely  affected  by  the  physician's  inability  to 
locate  films  and/or  interpretation  reports,  or  the  lack  of 
distribution  copies  available  to  physicians  or  in  the  patient  record. 
They  felt  that,  despite  improvements,  TRIRAD  had  not  totally 
eliminated  these  occurrences. 

The  timeliness  of  reports  to  the  Emergency  Room  was  also  explored 
through  interviews.  During  the  baseline  period,  ER  staff  reported 
turnaround  time  to  be  satisfactory.  During  July  1982,  six  months 
post-installation,  in  an  effort  to  improve  service,  the  ER  staff  had 
conducted  a  study  of  the  time  from  patient  departure  from  the  ER  to 
Radiology  to  the  return  of  patient  and  handwritten  X-ray  report  to  the 
ER  and  compared  results  with  two  civilian  teaching  hospitals  in 
Washington.  The  study  showed  an  average  turnaround  time  of  78  minutes 
in  contrast  to  average  times  of  30  minutes  and  5&  minutes  at  the  other 
two  hospitals.  The  ER  staff  identified  two  factors  contributing  to 
delays  In  Radiology:  that  ER  patients  did  not  receive  priority  in 
having  examinations  taken  and  that  the  time  to  retrieve  prior  films 
from  the  film  library  had  increased  since  the  baseline.  The  latter 
resulted  from  procedures  for  departing  patients  on  TRIRAD  that 
required  matching  new  films  and  prior  films  before  departing  the 
patient  and  sending  the  films  to  the  reading  room.  By  October,  nine 
months  post-installation,  both  situations  had  changed,  with  the  result 
that  ER  staff  were  considerably  more  satisfied  with  turnaround  time. 

Surgeons  and  an  anesthesiologist  cited  difficulties  in  both  the 
baseline  and  post-implementation  periods  for  obtaining  pre-operative 
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results  sufficiently  In  advance  to  resolve  questions  raised  by  the 
Radiology  report.  They  described  Instances  of  having  to  cancel 
surgery  because  of  Radiology  reports  read  and/or  delivered  the  morning 
of  surgery,  rather  than  during  the  late  afternoon  on  the  day  before, 
when  there  was  still  time  to  call  in  a  consultant  and  resolve  any 
questions. 

Attending  physicians  in  all  areas  noted  that  they  often  obtained 
reports  from  the  TRIRAD  CRT  terminal.  They  reported  using  this  means 
to  obtain  reports,  to  locate  files,  or  to  ascertain  the  status  of 
patient  examinations  not  yet  reported.  The  interviewees  believed  that 
access  to  reports  and  report  status  via  TRIRAD  was  proving  to  be  one 
of  the  major  benefits  of  the  system  to  attending  physicians. 

b.  Film  Access  and  Borrowing 

On  questions  relating  to  film  access  and  borrowing,  the 
physicians  interviewed  indicated  that  they  were  somewhat  dissatisfied 
in  the  post-implementation  period,  whereas  in  the  baseline  period  they 
indicated  that  they  were  very  dissatisfied.  Their  increase  in 
satisfaction  (or  decrease  in  dissatisfaction)  is  related  to  the 
improvement  in  locating  current  films  in  the  department,  aided  by  the 
file  tracking  features  of  TRIRAD. 

Interviewees  Indicated  a  desire  for  some  improvements  in 
procedures  for  borrowing  films  that  were  instituted  with  the 
installation  of  TRIRAD.  They  cited  difficulties  checking  out  a  large 
number  of  films  (e.g.,  for  a  teaching  conference).  They  reported  that 
the  bar-code  readers  did  not  always  accept  the  inputs  and  multiple 
strokes  of  the  light  pen  were  required  for  each  input,  resulting  in 
lengthy  check-out  procedures.  TRIRAD  was  designed  to  identify 
delinquent  borrowers.  However,  when  films  were  returned  to  Radiology, 
the  return  was  not  being  credited  to  the  physician's  borrowing  queue 
(due  to  a  software  error).  As  a  result,  the  original  plan  to  use  this 
queue  to  notify  borrowers  which  films  were  delinquent,  e.g.,  more  than 
one  week  overdue,  could  not  be  carried  out. 

A  further  source  of  dissatisfaction  with  the  loan  system  during 
post-implementation  was  that  loans  could  only  be  made  to  physicians 


90 


and  not  Co  clinics  or  wards.  This  approach  was  instituted  so  that  an 
individual  would  be  responsible  for  the  films.  However,  typically, 
the  day  before  a  clinic  was  held,  films  were  checked  out  for  all  the 
patients  to  be  seen  on  the  following  day.  This  could  amount  to  dozens 
of  files  for  patients  seeing  ten  different  physicians.  Since  signing 
ouc  each  film  to  the  actual  physician  was  found  to  be  cumbersome,  some 
clinics  adopted  the  practice  of  signing  out  all  films  for  the  clinic 
to  one  of  the  physicians.  This  practice  was  decreasing  the 
Department's  ability  to  track  films  loaned  out  and  to  attribute 
outstanding  loans  to  an  individual. 

The  staff  in  the  ER  expressed  particular  dissatisfaction  in  both 
study  periods  about  their  access  to  films.  This  steins  from  the 
requirement  that  the  ER  staff  come  to  Radiology  to  sign  out  each  file 
individually.  These  policies  have  been  in  effect  since  the  baseline 
period. 

c.  Other  Issues 

The  attending  physicians  interviewed  felt  that  the  quality  of  the 
films  and  the  accuracy  of  the  Radiology  reports  were  equal  in  baseline 
and  post-implementation  periods;  satisfaction  in  both  cases  was 
considered  good  and  the  quality  of  service  consistent  with  other 
teaching  facilities  where  Radiology  residents  are  responsible  for  some 
of  the  reading. 

Access  to  Radiology  consultations  was  generally  felt  to  be  useful 
and  a  satisfactory  component  of  the  overall  Radiology  Service. 

On  questions  relating  to  ease  of  ordering  examinations, 
interviewees  were  generally  satisfied  with  the  process  for  both 
routine  and  STAT  examinations. 

Attending  physicians  noted  an  increase  in  patient  waiting  tine, 
particularly  in  Main  Radiology.  These  increases  had  become  evident  to 
them  largely  from  patient  complaints,  especially  when  patients  had  to 
wait  for  wet-reading  reports.  Patient  waiting  time  was  not  felt  to  be 
a  general  problem  during  the  baseline  period. 
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V.  DISCUSSION  OF  RESULTS 


A.  INTRODUCTION 

This  chapter  reviews  the  results  of  quantitative  studies, 
surveys,  and  interviews  and  discusses  the  information  gained  from  the 
experience  with  TRIRAD  at  NHB  from  several  perspectives.  First,  each 
of  the  goals  and  objectives  established  by  the  TRIMIS  Medical  Review 
Group  is  examined.  Actual  pertinent  experience  documented  in  the 
evaluation  is  reviewed  and  the  influence  of  TRIRAD  and  the 
significance  of  changes  are  discussed.  Following  the  discussion  of 
system  goals  and  objectives,  two  other  topics  are  addressed:  the 
experience  with  reporting  assistance  for  radiologists  and  staff 
acceptance  of  TRIRAD. 

B.  GOALS  AND  OBJECTIVES 

1.  Goal  1:  To  Provide  Accurate,  Timely  Radiology  Reports 

Timeliness  of  interpretation  reports  was  an  acknowledged  problem 
at  NHB  before  the  installation  of  TRIRAD.  When  surveyed  during  the 
baseline  period,  staff  expressed  high  expectations  for  TRIRAD  in 
reducing  the  time  to  produce  a  report.  In  fact,  respondents  rated 
their  expectations  for  this  change  at  a  higher  level  than  for  any 
other  change.  Because  of  this  and  the  importance  of  timeliness  to 
radiology  services,  considerable  effort  in  the  evaluation  was  devoted 
to  documenting  changes  in  report  turnaround  time  in  the  various 
sections  of  the  Radiology  Department.  Staff  perceptions  regarding 
changes  and  the  role  of  TRIRAD  in  these  changes  were  also  solicited. 

Since  the  installation  of  TRIRAD,  the  turnaround  time,  as 
measured  between  patient  arrival  in  Radiology  until  the  final  written 
report  is  available  for  distribution,  has  decreased  substantially  in 
many  sections  of  the  department.  Additionally,  whereas  during  the 
baseline  study  many  final  reports  were  handwritten,  nearly  all  reports 
are  now  printed,  which  results  in  more  legible  reports.  These  two 
changes  must  be  considered  together  in  order  to  interpret  the 
significance  of  improved  timeliness. 
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Main  Radiology  films  were  Interpreted  in  either  the  Wet  Reading 
Room  or  the  Inpatient  Reading  Room.  In  the  Wet  Reading  Room,  reports 
were  all  handwritten  during  the  baseline  period  and  turnaround  time 
averaged  46  minutes.  A  portion  of  these  reports  required  a  wet 

reading;  however,  many  routine  reports  were  also  interpreted  in  this 
room  in  order  to  take  advantage  of  the  faster  turnaround  time  to  be 
gained  from  handwritten  reporting.  Approximately  95%  of  all  reports 
in  Main  Radiology  were  processed  in  this  manner.  A  small  number  of 
reports,  5%,  were  interpreted  in  the  Inpatient  Reading  Room.  These 

reports  were  dictated  and  transcribed  and  required  an  average  of  16 

days  for  completion. 

During  the  post-implementation  period,  only  6%  of  the  final 
reports  in  the  Wet  Reading  Room  were  handwritten.  All  other  reports 
were  printed  and,  of  these,  32%  also  received  a  preliminary 

handwritten  report.  In  Main  Radiology,  the  turnaround  time  for 
printed  reports  averaged  4  days  and  handwritten  reports  averaged  63 
minutes. 

In  Orthopedic  Radiology,  where  all  reports  were  handled  as  "wet 
reads,"  final  handwritten  reports  continued  to  serve  as  the  principal 
report,  and  turnaround  time  Increased  from  39  minutes  to  56  minutes. 
Following  implementation  of  TRIRAD,  reports  were  also  entered  into  the 
computer,  6%  by  bar  coding  and  94%  by  transcription  from  the 
»  handwritten  report.  These  printed  reports  were  distributed  within  4 

days  on  average. 

In  the  baseline  period,  reports  in  all  other  sections  were 
»  dictated  and  typed  and  this  practice  continued  into  the  post-implemen¬ 

tation  period.  Average  turnaround  time  in  these  areas  was  reduced 
from  a  range  of  15-43  days  in  the  baseline  period  to  5-14  days  in  the 
,  post-implementation  period. 

The  analysis  of  incomplete  reports  between  March,  August,  October 
and  No\’ember,  1982  (2,  7,  9,  and  10  months  post-installation)  clearlv 
shows  the  progression  of  improvement  in  the  Department's  ability  to 
reduce  the  backlog  of  reports  and  to  improve  the  timeliness  of  report 
completion.  For  all  Radiology  sections  summed  together,  the  length  of 
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time  required  to  complete  reports  for  more  than  50%  of  the  procedures 
for  a  given  day  has  steadily  decreased.  In  March,  approximately  16 
days  elapsed  before  50 %  of  the  reports  were  completed.  (During  this 
month.  Orthopedic  Radiology  was  just  beginning  to  enter  reports  into 
the  system;  previously  these  were  only  handwritten.)  In  August, 
reports  were  produced  for  50%  of  the  procedures  within  4  days.  In 
October  and  November,  this  time  was  reduced  to  3.5  days.  These 
figures  correlate  with  an  average  turnaround  time  from  examination 
until  report  printing  of  35  days. 

The  Radiology  Department  staff  were  surveyed  regarding  their 
satisfaction  with  the  timely  availability  of  reports  and  test  results. 
Ninety-two  percent  of  radiologists  were  dissatisfied  and  half  of  these 
responded  as  "very  dissatisfied"  concerning  a  baseline  question  on  the 
timely  availability  of  test  results;  50%  of  all  residents  were 
dissatisfied  and  half  of  these  were  "very  dissatisfied".  The 
remaining  radiologists  and  residents  answered  that  they  were 
"neutral,"  and  no  staff  indicated  that  they  were  satisfied  in  the 
baseline  period.  In  responding  to  the  post-implementation  survey,  40% 
of  radiologists  and  40%  of  residents  indicated  satisfaction  with  the 
timely  availability  of  current  reports  and  no  staff  indicated 
dissatisfaction. 

Results  of  interviews  with  attending  physicians  further  support 
these  findings.  These  physicians  were  very  dissatisfied  with  the 
timeliness  of  reports  during  the  baseline  period  for  both  inpatients 
and  outpatients.  During  post-implementation  interviews,  they  noted  a 
decrease  in  report  turnaround  time  and  indicated  that  reports  were 
more  often  found  in  both  inpatient  and  outpatient  records  when  needed. 

Attending  physicians  felt  that  more  improvements  in  report 
timeliness  were  needed.  Quantitative  data  show  that,  for  inpatient 
reports,  only  33%  of  the  patients  were  still  in-house  on  the  day  that 
the  hard-copy  report  was  distributed.  Attending  physicians  noted  a 
continued  reliance  on  Informal  mechanisms  (such  as  going  to  the 
Radiology  Department)  for  obtaining  the  information  contained  in  a 
Radiology  report.  They  felt  that  the  access  to  reports  on  the  TRIRAD 
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terminal  was  particularly  beneficial  for  retrieving  report  results  by 
either  telephone  inquiry  or  going  to  the  department  personally.  As 
soon  as  they  were  entered  into  the  system,  results  became  accessible, 
even  prior  to  radiologist’s  approval  or  printing  and  distribution. 

The  reasons  for  the  reduction  experienced  in  report  turnaround 
time  are  believed  due  primarily  to  a  commitment  on  the  part  of 
Radiology  management  to  make  some  changes  in  operating  and  staffing  in 
order  to  improve  turnaround  time  rather  than  due  solely  to  TRIRAD. 
The  most  important  change  was  in  bringing  transcription  services 
within  the  Department  by  adding  two  transcriptionists  to  the  Radiology 
Department  staff  rather  than  using  outside  services.  This  change 
speeded  up  individual  segments  in  the  report  turnaround  cycle. 

During  the  baseline  period,  reports  were  dictated  approximately  4 
days  after  examination.  One  day  of  the  total  turnaround  time  was 
spent  sending  dictation  tapes  to  transcription  and  one  day  was  spent 
returning  the  typed  reports  to  Radiology;  these  times  were  substan¬ 
tially  longer  when  transcriptions  were  done  by  an  outside  typing 
service.  Once  the  transcriptionists  were  located  within  Radiology, 
reports  tended  to  be  dictated  the  same  day  and  radiologists  often 
hand-delivered  dictation  tapes  of  handwritten  notes  to  the  transcrip¬ 
tionists.  Also  when  the  transcriptionist  entered  the  reports  directly 
into  the  system,  they  were  available  immediately  for  approval  and 
signature. 

In  addition  to  improved  transcription,  a  reduction  in  the  volume 
of  Radiology  examinations  during  the  summer  of  1982  helped  the 
transcriptionists  to  catch  up  on  the  backlog. 

Aside  from  management  changes  that  coincided  with  the 
implementation  of  TRIRAD,  the  specific  contribution  of  the  system  to 
improved  report  timeliness  appears  to  lie  in  eliminating  the  need  for 
transcription.  The  1751  of  reports  that  were  bar-coded  reduced  the 
volume  of  reports  to  be  typed.  Bar-coded  reports  in  Main  Radiology 
were  created  immediately  on  the  system  and  may  have  contributed  to 
lowering  the  average  turnaround  time  in  Main  Radiology  if  they  were 
also  approved,  printed  and  distributed  quickly. 
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In  interpreting  the  significance  of  changes  in  report  timeliness, 
it  is  important  to  consider  the  mix  in  types  of  reports  prepared  in 
the  baseline  and  post-implementation  periods.  In  Main  Radiology  most 
reports  were  handwritten  during  the  baseline  period  in  order  to  avoid 
the  lengthy  turnaround  time  to  prepare  a  typed  report;  nearly  all 
reports  in  Main  Radiology  during  the  post-implementation  period  were 
printed.  In  Orthopedic  Radiology  all  reports  were  handwritten  in  both 
periods  because  of  the  need  for  immediate  results.  Using  the  TRIRAD 
system,  many  of  these  were  also  entered  into  the  system  so  that  the 
report  could  be  printed  for  the  medical  record  and  could  be  retrieved 
from  the  system  at  a  later  time;  of  these,  a  few  were  entered  by  bar 
coding  whereas  most  were  transcribed  from  the  handwritten  report. 
Thus,  in  these  two  sections  there  was  a  shift  in  the  type  of  report 
being  prepared  and  in  the  type  of  service  offered,  along  with  a  change 
in  timeliness. 

The  improvement  in  timeliness  of  interpretation  reports  noted  in 
studies  conducted  within  the  Radiology  Department  was  confirmed  by  the 
attending  physicians  interviewed.  They  reported  that  they  were  more 
likely  to  find  interpretation  reports  in  patient  records,  though 
ideally  they  desired  further  improvements.  A  comparison  of  report 
turnaround  time  criteria  (representing  a  consensus  of  Radiology  staff) 
with  the  data  from  the  baseline  and  post-implementation  periods  add 
further  support  to  these  findings.  These  criteria  suggest  that 
Radiology  staff  considered  that  a  turnaround  time  of  24-48  hours  was 
acceptable  depending  upon  the  type  of  examination;  a  time  period  cf 
8-24  hours  was  preferred.  Since  turnaround  time  for  typed/printed 
reports  was  decreased  to  an  average  of  5  days  in  the  post-implemen¬ 
tation  period,  an  increase  in  satisfaction  among  Radiology  staff  and 
attending  physicians  is  consistent  with  this  positive  change. 
However,  with  the  gap  between  5  days  and  the  criterion  of  one  or  two 
days,  it  is  understandable  that  staff  would  like  to  see  further 
improvements. 

Written  reports  are  not  the  only  means  whereby  attending 
physicians  receive  radiology  results.  In  the  Inpatient  Reading  Room, 
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for  example,  films  are  scored  on  multi-film  viewers  throughout  the 
patient’s  stay.  Several  clinical  services  conduct  Radiology  rounds  at 
these  viewers  and  may  review  all  films  taken  on  that  day  on  their 
patients.  Attending  physicians  also  discuss  the  interpretation  of 
films  with  radiologists  and  retrieve  reports  from  the  TRIRAD  system. 
Improvements  in  these  informal  mechanisms  for  reporting,  especially 
the  increase  in  ease  and  timely  availability  of  retrieving  reports, 
were  considered  by  attending  physicians  to  be  the  most  significant 
benefit  to  staff  outside  of  Radiology. 

With  respect  to  report  availability  in  records,  the  attending 
physician  is,  however,  somewhat  removed  from  the  part  of  the  radiology 
process  affected  by  TRIRAD.  The  processes  for  (I)  radiology  report 
distribution,  (2)  delivery  to  the  appropriate  ward,  clinic,  or  record 
room,  and  (3)  filing  in  the  appropriate  record,  were  only  indirectly 
affected  by  TRIRAD,  and,  therefore,  more  timely  availability  of  the 
report  for  distribution  from  the  Radiology  Department  alone  would  not 
be  expected  to  address  delays  in  these  other  segments  of  the  total 
turnaround  cycle. 

The  analysis  of  complete  and  incomplete  reports  shows  a 
continuous  progression  of  improvement  in  the  Department's  ability  to 
complete  reports  more  quickly.  This  steady  improvement  from  March 
through  November,  1982  appears  likely  to  continue  since  changes  were 
still  being  made  during  the  post-implementation  data-collection 
period.  For  example,  in  Nuclear  Medicine  radiologists  changed  from 
dictation  on  tape  (for  later  transcription) ,  to  direct  dictation  where 
the  transcriptionist  entered  the  report  immediately.  In  Computerized 
Tomography,  the  radiologists  established  a  schedule  for  interpretation 
and  reporting  to  replace  the  unscheduled  reading  sessions.  In  both  of 
these  sections,  these  changes  resulted  in  a  decrease  in  turnaround 
cine  from  the  beginning  to  the  end  of  the  3-week  data  collection 
period. 

When  surveyed  during  the  post-implementation  period.  Radiology 
staff  felt  strongly  that  benefits  would  continue  to  accrue  in  the 
future.  Radiology  staff,  when  surveyed  during  the  post-implementation 
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period,  reporced  chat  telephone  Inquiries  for  reports  had  decreased. 
This  appears  to  be  a  reflection  of  the  fact  that  attending  physicians 
had  to  obtain  fewer  reports  by  telephone  because  they  received  formal 
printed  reports  in  a  more  timely  manner  and  were  provided  with 
increased  access  to  reports  via  the  TRIRAD  terminals  in  the 
Department. 

Information  regarding  accuracy  of  interpretation  reports  was 
gained  by  survey  and  interviews.  Radiology  Department  staff  were 
asked  in  the  post-implementation  questionnaire  whether  TRIRAD  had 
Increased  the  accuracy  of  Radiology  reports.  Though  two  radiologists 
surveyed  felt  that  accuracy  had  increased,  the  remainder  were  neutral 
(3)  or  felt  that  accuracy  had  not  been  affected  (7).  Residents  who 
responded  to  the  survey  believed  four  to  one  that  accuracy  had 
increased.  Interviews  with  attending  physicians  indicated  that  they 
viewed  the  accuracy  of  reports  as  satisfactory  both  before  and  after 
TRIRAD  installation  and  no  changes  were  noted. 

The  difference  in  responses  between  radiologists  and  residents  is 
interesting.  The  two  groups,  or  individual  physicians  within  each 
group,  may  in  fact  be  responding  to  different  interpretations  of  this 
question.  TRIRAD  does  not  directly  contribute  to  the  accuracy  of  the 
interpretation  itself,  i.e.,  TRIRAD  does  not  aid  the  radiologist  in 
seeing  more  abnormalities  nor  in  interpreting  their  significance.  If 
the  question  of  accuracy  is  interpreted  in  this  way  a  response  of  "no 
benefit"  or  "neutral"  appears  a  consistent  response.  TRIRAD  can, 
however,  contribute  to  the  completeness  of  the  printed  report,  and  to 
the  accuracy  of  the  information  contained  in  the  printed  report,  e.g., 
the  correct  date,  examination  type,  patient  identification,  clinical 
information,  and  patient  location.  Together  these  contribute  to 
Increased  accuracy  of  the  information  contained  in  the  report. 

2.  Goal  2:  To  Collect  Management  and  Workload  Data  for  Use  in 
Optimizing  Radiology  Resources 

The  Administrative  Reporting  and  Statistics  module  was  intended 
to  provide  access  to  information  from  the  patient  data  base  for  a 
variety  of  management  purposes.  At  the  time  of  the  post-implemer- 
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Cation  study,  this  module  of  TRIRAD  was  in  the  process  of  being  in¬ 
stalled,  and,  therefore,  the  experience  with  management  information 
was  extremely  limited. 

The  basic  data  for  radiology  management  reports  were  captured 
from  schedules.  They  were  supplemented  with  information  entered  into 
the  system  at  patient  check-in,  the  time  of  examination,  the  date  of 
reporting,  etc.  By  the  time  of  the  post-implementation  study,  these 
data  elements  were  all  available  in  the  system  and  some  information 
based  on  these  elements  (e.g.,  report-completion  status)  could  be 
reported  on  demand.  However,  because  the  ability  to  manipulate  data 
and  produce  specialized  reports  (e.g.,  list  incomplete  reports 
transcribed  but  awaiting  approval)  was  limited,  use  of  the  information 
available  from  TRIRAD  for  management  purposes  was  at  a  very 
preliminary  stage. 

Some  management  reports  were  available  as  part  of  TRIRAD 
functions  that  had  already  been  implemented.  However,  these  reports 
were  not  designed  to  be  assembled  and  printed  in  a  concise  format. 
For  example,  during  several  implementation  monitoring  visits  and 
during  post-implementation  data  collection,  daily  logs  were  printed  in 
order  to  obtain  the  data  needed  to  prepare  the  graphs  of  complete  and 
incomplete  reports  presented  in  Chapter  IV.  The  daily  totals  of 
complete  and  Incomplete  reports  were  all  that  were  needed.  However, 
in  order  to  obtain  these  numbers,  a  complete  listing  of  all  patients 
examined  was  obtained  and  then  a  manual  count  was  made.  Printing 
these  reports  for  1  month  of  day  logs,  approximately  7000 
examinations,  required  2  hours.  So  while  this  information  was  quite 
valuable  once  obtained,  it  required  substantial  effort  to  produce  and 
analyze  and  thus  was  not  used  routinely  in  the  Department  as  a 
management  tool. 

When  the  management  information  capabilities  are  fully 
implemented,  TRIRAD  is  expected  to  provide  much  of  the  information 
needed  to  complete  management  reports  prepared  in  the  Department  such 
as  for  the  Uniform  Chart  of  Accounts.  Use  of  computer-tabulated 
workload  data  in  place  of  information  compiled  manually  from  paper 
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records  should  Increase  Che  accuracy  of  the  data  and  reduce  the  time 
required  to  perform  these  reporting  functions.  TRIRAD  is  also 
expected  to  make  available  management  information  that  was  infeasible 
to  compile  by  manual  methods  (e.g.,  producing  a  list  of  files  Co  be 
purged  from  the  film  library  on  patients  who  have  not  had  an 
examination  in  the  Department  for  5  years). 

3.  Goal  3:  To  Improve  Availability  of  Patient-Specific  Historical 
Radiological  Reports  and  Films 

In  the  baseline  process,  interpretation  reports  and  films  were 
filed  together  in  the  film  file  folder.  Therefore,  the  availability 
of  both  reports  and  films  depended  upon  the  ability  of  the  Department 
to  manage  its  files.  Once  TRIRAD  had  been  implemented,  interpretation 
reports  were  stored  on  the  system,  and  the  copy  of  the  interpretation 
report  filed  with  the  film  was  no  longer  the  only  record  available  in 
the  Department.  Film  file  management  practices  were  also  changed  by 
the  introduction  of  the  film  file  tracking  features  of  TRIRAD,  which 
partially  automated  the  recordkeeping  aspects  of  file  management. 
Therefore,  whereas  the  issues  relating  to  report  availability  and  film 
availability  were  the  same  in  the  baseliru  study,  the  TFIRAD  functions 
affecting  the  availability  of  these  two  types  of  historical 
information  were  different  and  the  impacts  on  availability  need  to  be 
considered  separately. 

Report  and  film  availability  were  included  in  the  surveys  of 
Radiology  Department  staff  and  were  explored  in  interviews  vith 
attending  physicians.  In  addition,  data  concerning  film  availability 
were  collected  in  all  reading  rooms  where  radiologists  were  observed 
(Main  Radiology,  GI  Radiology,  Orthopedic  Radiology,  Nuclear  Medicine, 
Computerized  Tomography  and  Ultrasonography) . 

Survey  questions  relating  to  historical  reports  and  films 
addressed  two  components  of  availability:  timeliness  of  availability 
and  ease  of  obtaining  the  information.  The  baseline  survey  indicated 
that  Radiology  Department  staff  overall  were  slightly  dissatisfied 
with  both  the  timeliness  of  historical  record  availability  (film  and 
report  filed  together)  and  the  ease  of  obtaining  historical  records. 
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Radiologists  as  a  group  indicated  that  they  were  less  satisfied  with 
both  aspects  (with  an  overall  mean  rating  indicating  they  were 
"somewhat  dissatisfied"). 

In  the  post-implementation  survey,  overall  satisfaction  of  staff 
with  timely  availability  of  historical  records  was  improved. 
Radiologists,  though  still  less  satisfied  than  other  staff,  also 
appeared  to  be  slightly  more  satisfied  than  during  the  baseline 
period.  This  change  is  believed  to  result  from  the  capability  to 
retrieve  interpretation  reports  via  the  system.  Attending  physicians 
noted  that  this  capability  was  the  single  most  important  benefit  of 
TRIRAD. 

Surveys,  interviews,  and  quantitative  studies  all  indicated  less 
improvement  in  the  availabiity  of  historical  films.  Staff 
satisfaction  with  availability  of  films  remained  essentially  unchanged 
between  the  baseline  and  post-implementation  surveys,  though  their 
satisfaction  with  the  ease  of  obtaining  them  increased  somewhat. 

Analysis  of  the  number  of  prior  film  files  that  were  available  in 
the  reading  room  for  use  by  the  radiologist  in  interpretation  showed 
that  there  was  a  prior  file  for  412  of  the  interpretations  in  the 
baseline  period  and  for  32%  in  the  post-implementation  period.  There 
was  also  a  sizeable  portion  of  the  readings  for  which  the  data 
collector  was  not  able  to  determine  whether  there  was  a  prior  film 
(17%  in  the  baseline  and  18%  in  the  post-implementation  period).  The 
conclusion  is  that  availability  of  prior  films  did  not  change 
significantly.  So  that  while  satisfaction  had  increased  slightly,  no 
more  historical  film  files  were  actually  present  during  reading. 
These  findings  support  the  conclusion  that  obtaining  films  was 
slightly  easier,  but  no  more  films  were  available  for  comparison. 

The  extent  to  which  TRIRAD  functions  replaced  manual  processes 
differed  for  historical  films  and  reports,  and  this  is  the  likely 
explanation  for  the  different  impacts  on  availability.  Film 
management  functions,  such  as  file  tracking  and  label  generation,  were 
automated  but  films  were  still  filed  and  retrieved  manually  in  the 
film  library. 
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Further  improvements  to  be  gained  through  automated  retrieval  of 
historical  films  await  the  development  of  new  technologies  that  will 
eliminate  the  need  for  film-file  libraries.  One  such  technology  is 
digital  radiography,  which  promises  to  obtain  and  store  radiographic 
images  digitally.  With  this  technology,  films  would  no  longer  be  used 
and  radiologists  would  view  Images  on  video  display  terminals.  These 
technologies  already  exist  but  are  not  yet  in  widespread  use  due,  in 
part,  to  the  high  cost  of  the  hardware  and  the  high  cost  of  the 
storage  necessary  to  file  the  images. 

4.  Goal  4:  To  Make  Maximum  Use  of  Resources,  Equipment  and  Staff 
Through  Efficient  Scheduling 

This  goal,  as  formulated  by  the  Medical  Review  Group,  includes 
inter-related  Improvements  in  the  efficiency  of  scheduling  and  the 
ability  of  the  Department  to  utilize  its  resources. 

Scheduling/reception  functions  were  included  in  the  staff  surveys 
conducted  before  and  after  TRIRAD  installation. 

The  baseline  survey  included  a  question  regarding  expected 
changes  in  scheduling.  All  groups  of  staff  expected  that  TRIRAD  would 
improve  scheduling.  At  the  time  post-implementation  data  were 
collected,  the  optimal  scheduling  function  of  TRIRAD  was  not  fully 
operational  due  to  software  problems.  The  manual  scheduling  module 
functioned  well  and  was  used  in  all  sections  of  Radiology.  This  meant 
that  the  system  was  producing  schedules  and  performing  functions  such 
as  checking  for  conflicts  in  scheduled  examinations  and  displaying 
schedule  rosters  ir.  order  to  allow  the  scheduler  to  select  an 
available  appointment.  When  surveyed  during  the  post-implementation 
period,  Radiology  staff  felt  that  the  use  of  TRIRAD  had,  in  fact, 
resulted  in  improved  scheduling.  Thirty-two  staff  felt  that  there  had 
been  benefits  to  scheduling  while  only  six  felt  that  no  benefits  had 
been  realized.  The  distribution  of  responses  was  similar  among 
radiologists,  residents  and  technicians  and  other  staff. 

Work  sampling  was  conducted  at  the  reception  areas  in  Main 
Radiology  and  Imaging.  The  percentage  of  time  devoted  to  each 
reception  activity  or  function  was  calculated.  In  both  Main  Radiology 
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and  Imaging  Che  percentage  of  clme  spent  on  scheduling  and  receiving 
of  patients  increased  from  162  in  the  baseline  period  to  212  during 
the  post-implementation  period.  This  increase  appears  to  be  due 
directly  to  the  time  spent  entering  additional  patient  data  in  order 
to  develop  more  complete  patient  examination  histories  and  data  files. 
This  Information  was  then  available  to  all  other  users  of  the  system. 

A  positive  change  in  scheduling,  due  directly  to  TRIRAD,  appears 
certain  to  have  occurred.  However,  evidence  for  an  influence  of  this 
change  on  resource  utilization  appears  mixed. 

The  post-implementation  survey  of  Radiology  Department  staff 
included  questions  related  to  the  impact  of  TRIRAD  on  the  efficiency 
and  economy  of  delivering  radiology  services.  A  large  number  of  staff 
(27  or  512  of  the  respondent  sample)  reported  that  TRIRAD  had  made 
their  work  more  efficient  and  that  TRIRAD  saved  time  (23  or  432  of  the 
respondent  sample).  The  conclusion,  based  on  this  information,  seems 
to  be  that  increased  efficiency  was  realized. 

However,  the  staff  surveyed  did  not  believe  that  TRIRAD  had 
reduced  overall  personnel  requirements  nor  the  cost  of  radiology 
services.  (This  was  consistent  with  their  expectations  concerning 
personnel  requirements  and  costs.)  The  examination  of  Department 
budgets  also  indicated  no  substantial  shifts  in  staffing  or  other 
resources  due  to  TRIRAD,  at  least  during  the  first  year  of  system 
operation.  Thus  the  impact  of  TRIRAD  on  resource  utilization  appears 
to  have  been  in  some  improvements  in  efficiency,  but  these 
efficiencies  were  not  of  a  magnitude  that  the  economy  of  service 
delivery  could  be  measurably  improved. 

The  specific  role  of  TRIRAD  scheduling  functions  In  increasing 
the  efficiency  of  operations  should  have  been  reflected  in  decreased 
patient  waiting  and  examination  time.  As  discussed  in  the  subsequent 
section  dealing  with  the  goal  related  to  this  issue,  the  evidence  is 
also  somewhat  mixed. 

In  five  sections  of  the  Department,  these  times  were  found  to 
increase  somewhat.  In  the  three  sections  where  tine  reductions  were 
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noted.  It  appears  that  these  reductions  were  due  to  reasons  other  than 
Improved  scheduling. 

Thus  though  it  would  appear  that  scheduling  did  improve,  and  that 
Department  efficiency  was  enhanced  in  some  areas,  the  relationship 
between  these  two  changes,  as  anticipated  by  the  Medical  Record  Review 
Group  has  not  been  demonstrated.  (A  possible,  though  not 
well-substantiated  benefit,  of  patient  registration  and  scheduling 
using  TRIRAD  is  more  reliable  capture  of  workload  performed  in  the 
Department.) 


5.  Goal  5:  To  Reduce  Patient  Waiting  and  Processing  Time  Due  to 


As  was  the  case  for  Goal  4,  the  Medical  Review  Group  anticipated 
that  improved  scheduling  would  result  in  other  changes,  in  this  case  a 
reduction  in  the  elapsed  time  from  patient  arrival  in  the  Department 
until  completion  of  the  examination  and  departure  of  the  patient. 

Quantitative  studies  conducted  during  the  evaluation  included 
measurements  of  changes  in  patient  waiting  and  processing  time.  The 
data  indicate  that  these  times  Increased  somewhat  in  five  sections  and 
decreased  in  three  other  sections  of  the  Department.  As  discussed 
previously  (Goal  4),  the  TRIRAD  scheduling  functions  had  not  been 
fully  implemented  when  post-implementation  data  were  collected. 
Nevertheless,  staff  in  the  Department  believed  that  scheduling  had 
improved.  In  order  to  elucidate  any  relationship  between  this 
improvement  and  patient  waiting  and  examination  time,  it  is  necessary 
to  consider  the  experience  in  the  individual  sections  of  the  Radiology 
Department. 

In  Main  Radiology,  the  increase  in  time  appears  to  be  related  to 
the  increase  in  elapsed  time  to  request  prior  film  files  and  match 
these  historical  records  to  the  current  films  prior  to  reading. 

In  GI  Radiology,  it  appears  that  the  examinations  performed 
during  the  post-implementation  period  required  more  time  to  complete 
on  average  than  those  during  the  baseline  and  does  not  appear  to  be 
due  to  changes  in  other  parts  of  the  process. 


In  Orthopedic  Radiology,  the  Increase  In  patient  waiting  and 
examination  time  appears  due  in  part  of  slower  processing  of  patients 
due  both  to  entering  the  additional  data  required  by  TRIRAD  and  to 
changes  in  the  Orthopedic  Clinic  independent  of  the  X-ray  functions. 

In  Urologic  Radiology  and  Special  Procedures,  process  time 
appears  to  have  decreased.  There  were  no  observed  changes  in  the 
process  to  account  for  these  changes.  Possibly  the  mix  of  examination 
times  changed  or  the  limited  observations  from  the  baseline  period 
were  not  representative  of  the  average  times. 

In  Nuclear  Medicine,  the  mix  of  examinations  changed.  During  the 
post-implementation  study  period  56Z  of  the  patients  cracked  had 
either  liver  scans  or  bone  scans  which  have  very  long  examination 
times.  These  two  examinations  represented  a  much  smaller  proportion 
of  the  baseline  sample. 

In  Computerized  Tomography,  unscheduled  patients  were  added  to 
the  schedule  more  often  in  the  post-implementation  period.  For 
example,  if  an  outpatient  who  was  scheduled  for  a  CT  scan  in  advance 
was  late  for  the  appointment,  CT  staff  filled  the  apparent  opening  in 
the  schedule  by  substituting  an  examination  on  an  inpatient.  On 
several  occasions  the  inpatient  was  sent  to  the  CT  section  and  the 
late  outpatient  also  arrived.  This  resulted,  in  effect,  with  two 
patients  scheduled  for  the  same  time  period,  and  thus  increased 
waiting  time  for  one  of  these  patients. 

In  Ultrasonography,  the  decrease  in  waiting  time  appears  due  to  a 
change  in  radiologist  staffing.  The  radiologist  during  the  baseline 
period  preferred  to  be  involved  in  taking  the  examination,  often 
obtaining  additional  views  after  the  technician  completed  the  routine 
views,  resulting  in  increased  waiting  and  examination  time  for  the 
patients.  The  radiologist  during  post-implementation  preferred  that 
the  technicians  perform  the  complete  study.  It  is  believed  that  these 
changes  in  staffing  and  procedures  account  for  the  measured 
differences  in  process  time. 

Overall,  only  in  Computerized  Tomography  did  the  actual  proce¬ 
dures  for  scheduling  change  substantially  in  the  post-implementation 
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period,  and  in  this  case,  the  changes  were  not  due  to  TRIRAD.  These 
observations,  combined  with  quantitative  data  to  support  the  reasons 
for  changes,  reduce  the  confidence  that  the  apparent  increases  or 
decreases  in  patient  processing  time  are  due,  to  a  measurable  degree, 
to  scheduling  improvements. 

6.  Goal  6:  To  Reduce  the  Number  of  Repeat  Exams  as  a  Result  of 
Improved  Film  File  Accessibility 

Others  have  reported  that  stand-alone  computer  support  to 

(3) 

radiology  departments  reduced  the  number  of  repeat  exams  .  As 
stated  by  the  Medical  Review  Group,  this  goal  of  reduced  repeat 
examinations  was  expected  to  result  from  improved  accessibility  of 
film  files. 

When  surveyed  in  the  baseline  period,  staff  expected  that  TRIRAD 
would  have  a  definite  impact  on  access  to  current  films.  Indeed, 
access  to  current  films  appears  to  have  increased.  The  film-tracking 
functions  of  TRIRAD  have  made  it  easier  to  obtain  films  from 
outpatients  taken  during  the  past  day  or  two  or  from  inpatients  taken 
during  the  present  hospitalization  period.  Regarding  the  ease  of 
obtaining  current  films  the  weighted  mean  response  improved  slightly 
from  -0.3  to  -*-0.1  from  the  baseline  to  post-implementation  periods, 
respectively.  To  a  somewhat  lesser  extent,  TRIRAD  reduced  the  time  to 
locate  these  films.  On  questions  relating  to  the  timely  availability 
of  current  films,  survey  respondents  gave  a  weighted  mean  response  of 
-0.2  (slight  dissatisfaction)  during  the  baseline  and  +0.2  (slight 
satisfaction)  during  the  post-implementation  period. 

However,  though  film  file  accessibility  appears  to  have  improved 
somewhat  for  current  films,  accessibility  appears  unchanged  for 
historic  films.  When  staff  were  surveyed  on  satisfaction  with  the 
timely  availability  of  historic  films,  respondents  indicated  a  nearly 
comparable  level  of  slight  dissatisfaction  in  both  the  baseline  (-0.3) 
and  post-implementation  (-0.2)  periods.  Quantitative  data  collected 
during  observations  of  radiologists  during  reading  showed  a  slight 
decrease  in  the  number  of  prior  files  available,  from  41%  in  the 
baseline  and  32%  in  the  post-implementation  period.  Rote  that  this 
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study  focused  on  the  availability  of  prior  films  at  the  time  of  the 
official  reading.  Mo  attempt  was  made  to  determine  which  current 
films  should  have  had  a  prior  film.  Accordingly,  data  were  collected 
during  observations  of  radiologists  while  reading  and  no  distinction 
was  made  for  first-time  examinations  for  patients  and  for  examinations 
for  which  prior  films  were  out  on  loan.  This  methodology  was 
consistently  followed  in  both  study  periods. 

Staff  gave  mixed  responses  on  whether  TRIRAD  had  reduced  the 
number  of  lost  files;  16  respondents  to  the  survey  felt  that  there  was 
a  benefit  while  17  felt  there  was  no  benefit.  In  contrast  to  these 
findings,  when  surveyed  during  the  baseline  period  regarding  their 
expectations  for  TRIRAD,  all  staff  groups,  and  in  particular 
radiologists  and  other  staff,  felt  strongly  that  TRIRAD  would  have  a 
definite  Impact  on  reducing  lost  files.  The  reasons  for  the  lack  of 
change  in  availability  of  historic  films  appear  to  be  two-fold. 
First,  file  retrieval  depends  to  a  significant  extent  on  film-library 
management.  TRIRAD  performed  record-keeping  functions,  and  this 
appears  to  have  improved  the  process  for  obtaining  films  from  the 
library.  The  need  for  manual  filing  and  retrieval  of  the  file  folder 
was  not  eliminated  and  consequently,  the  availability  of  historical 
film  files  did  not  change,  although  the  availability  of  current  films 
may  have  Increased.  Second,  TRIRAD  had  only  been  operational  for  nine 
months  at  the  time  of  the  post-implementation  study.  Nine  months  may 
not  have  been  a  sufficient  period  of  time  to  establish  an  adequate 
historical  data  base.  (Department  staff  felt  that  18  months  would 
have  been  better.) 

Questions  regarding  repeat  examinations  were  included  in 
post-implementation  surveys  of  Radiology  Department  staff.  Twenty 
(38%  of  survey  respondents)  staff  members  felt  that  TRIRAD  had  reduced 
the  number  of  unnecessary  repeat  examinations,  a  large  number  ( 1 8 > 
were  neutral,  and  only  13  (24%)  felt  that  no  change  had  occurred. 
These  responses  suggest  that  there  was  a  reduction,  at  least 
experienced  by  some  of  the  staff. 
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Given  Che  lack  of  discernible  change  in  film-file  accessibility, 
there  are  other  possible  bases  for  expecting  a  reduced  need  for  repeat 
examinations.  The  turnaround  time  for  reports  decreased  overall  in 
the  Department  (discussed  previously  under  Goal  1).  Perhaps  even  more 
significant  was  the  increased  availability  of  patient  information 
reported  by  both  Department  staff  and  attending  physicians  (discussed 
previously  under  Goal  3).  This  included  both  access  to  completed 
reports  stored  in  the  system  and  the  ability  to  ascertain  the  status 
of  examinations  not  yet  reported  out  of  the  Department.  An 
association  between  these  improvements  and  reduced  repeat  examinations 
seems  quite  likely. 

7.  Goal  7;  To  Reduce  Personnel  Time  Spent  in  Clerical  Transcriptions 

During  the  baseline  period,  transcription  of  radiology  reports 
was  performed  outside  of  the  Radiology  Department,  both  in  the 
hospital's  Central  Transcription  section  and  by  a  typing  service 
outside  the  hospital.  Observations  during  the  baseline  period 
indicated  that  this  process  was  very  inefficient  and  contributed 
significantly  to  the  long  turnaround  time  for  typed  reports.  As  a 
result,  radiologists  made  extensive  use  of  handwritten  reports,  which 
they  viewed  as  less  desirable  but  a  more  efficient  way  to  prepare 
reports  for  distribution  in  a  timely  manner. 

Following  TRIRAD  installation,  the  transcription  service  was 
changed;  two  typists  were  added  to  the  Radiology  Department  staff  and 
policy  was  established  that  all  transcription  would  be  done  within  the 
Department.  The  result  has  been  reduced  turnaround  time  for  reports, 
accompanied  by  increased  satisfaction  of  both  Radiology  Department 
staff  and  attending  physicians. 

The  most  significant  aspect  of  the  improved  transcription 
services  is  that  the  Department  is  now  able  to  produce  most 
interpretations  in  printed  rather  than  handwritten  form.  Whereas  in 
the  baseline  65%  were  handwritten,  only  35%  were  handwritten  during 
post-implementation.  The  ability  to  improve  services  in  this  manner 
and  produce  timely  reports  clearly  resulted  from  the  changes  in  the 
efficiency  of  transcription  services  that  accompanied  TRIRAD. 
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Because  of  the  shift  tovard  printed  reports,  more  staff  time  is 
being  devoted  to  transcription.  Rowever,  the  efficiency  of  the 
transcription  segment  of  the  process  has  clearly  been  improved. 
(Though  an  actual  comparison  of  labor  time  for  transcription  is 
precluded  by  the  absence  of  baseline  data.  Radiology  staff  believed 
that  the  power  typing  capabilities  of  TRIRAD  had  Increased  the 
efficiency  of  transcriptlonists.) 

Another  influence  of  TRIRAD  on  the  transcription  process  has  been 
the  added  capability  for  radiologists  to  produce  an  interpretation 
report  directly  and  without  transcription.  Observations  of  reporting 
methods  Indicate  the  17%  of  all  reports  prepared  in  the  Department 
used  bar  coding,  thus  averting  the  need  for  transcription.  By 
projecting  this  volume  of  reports  to  the  annual  workload  it  was 
calculated  that  nearly  500  hours  of  transcription  labor  per  year  might 
be  averted  through  the  use  of  bar  coding.  The  experience  at  NHB  with 
this  computer  assistance  to  radiologist  reading  and  report  preparation 
is  discussed  in  the  following  section. 

C.  READINC  ASSISTANCE 

1.  Introduction 

TRIRAD  gave  the  staff  radiologists  and  radiology  residents  at  NHB 
the  ability  to  produce  printed  interpretation  reports  directly  from 
input  via  a  bar-coding  system  or  an  optical  mark  reader  (OMR) .  This 
section  first  discusses  the  utilization  and  staff  acceptance  of  these 
reading  and  reporting-assistance  functions  at  NHB  and  then  explores 
the  potential  significance  of  this  type  of  computer  assistance  in 
reducing  the  turnaround  time  for  typed /printed  interpretation  reports. 

2.  Bar-Coding  System 

During  observations  of  radiologists  in  the  reading  rooms  the 
method  of  reporting  and,  in  the  post-implementation  period,  the  use  of 
computer  assistance  was  recorded.  Use  of  this  TRIRAD  function  was 
observed  in  Main  Radiology  and  Orthopedic  Radiology. 

In  the  Main  Radiology  Wet  Reading  Room,  bar  coding  was  used  to 
prepare  all  or  part  of  40%  of  the  interpretation  reports  in  the 
observation  sample.  All  of  these  bar-coded  reports  were  prepared  by  a 
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single  radiologist,  one  of  the  two  radiologists  who  developed  the 
bar-coding  system  being  used  at  NHB  and  an  enthusiastic  advocate  of 
this  TRIRAD  feature. 

Bar  coding  was  also  used  to  a  more  limited  extent,  62  of  all 
reports  observed,  in  Orthopedic  Radiology.  First,  a  handwritten 
report  was  prepared  and  then,  for  selected,  simple  interpretations, 
the  report  was  entered  by  bar  coding.  The  remaining  handwritten 
reports  were  delivered  to  transcription  to  be  entered  into  the  system. 
The  staff  radiologist  in  this  section  and  the  staff  radiologist  in  GI 
Radiology  both  have  indicated  that  they  would  use  bar  coding  for 
nearly  all  of  their  reports  if  intelligent  video  display  terminals 
were  installed  in  these  reading  rooms.  Bar  coding  was  not  used  in  any 
of  the  the  other  reading  rooms. 

The  data  obtained  on  radiologist  reading  times  suggest  that  the 
preparation  of  bar-coded  reports  in  Main  Radiology  required  less 
radiologist  time  than  did  dictated  reports.  Interpretation  of  this 
finding  is  made  difficult  by  the  fact  that  bar  coding  appeared  to  be 
used  generally  for  simpler  reports  and  dictation  for  more  complex 
reports.  Since  the  mix  of  examinations  in  the  observations  of  reading 
times  is  different  for  each  method,  the  apparent  time  differences 
measured  cannot  be  readily  interpreted. 

3.  Optical  Mark  Reader 

The  optical  mark  reader  (OMR),  also  known  as  Raport  II,  was  not 
used  during  the  post-implementation  data  collection  period.  The  OMR 
capability  was  made  available  at  NHB  5  months  after  the  beginning  of 
TRIRAD  Installation.  At  that  time  bar  coding  was  being  used,  and  this 
decreased  the  interest  in  an  additional  method  of  computer  assistance. 

Observations  and  discussion  with  radiologists  suggest  several 
reasons  why  this  feature  was  not  integrated  into  routine  reporting. 
First,  the  OMR  did  not  have  an  advocate  (as  did  bar  coding)  to 
encourage  the  use  of  the  method  within  the  Department  among  the  staff. 
Second,  the  OMR  forms,  when  read  by  the  scanner,  generated  an 
immediate  report  without  going  through  the  radiologist's  approval 
queue;  radiologist's  signature  was  made  by  marking  a  code  on  the  form. 
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Radiologists  were  very  reluctant  to  release  interpretation  reports  in 
this  manner,  particularly  since  a  slight  error  in  marking  the  form 
could  lead  to  an  incorrect  report.  The  verification  process  for 
bar-coded  and  dictated-and-transcribed  reports  gave  them  an 
opportunity  to  review  the  report  via  terminal  prior  to  release.  (It 
is  believed  that  OMR-generated  reports  could  be  added  to  the  approval 
queue  with  a  minor  change  in  software.) 

The  OMR  was  also  felt  to  be  more  limiting  than  the  bar-coding 
system  and  required  a  greater  change  in  how  radiologists  read  films. 
The  bar-coding  system  used  at  NHB  was  specifically  organized  to  follow 
a  well-accepted  approach.  Finally,  the  OMR  presented  an  additional 
system  to  learn  without  enough  apparent  advantages  to  motivate 
radiologists  to  become  fluent  in  the  use  of  the  mark-sense  forms. 

4.  Significance 

The  real  significance  to  radiologists  of  computer  assistance  in 
reading  films/images  and  preparing  interpretation  reports  may  be  that 
it  eliminates  the  need  for  transcription  and  the  attendant  delays  in 
reporting  the  interpretation  out  of  the  department.  Observations  of 
report  turnaround  time  show  that  the  transcription  segment  contributed 
1  to  3  weeks  to  the  total  time  in  the  baseline  and  less  than  one  day 
tc  the  time  for  transcribed  reports  in  the  post-implementation  study. 
By  contrast,  in  Main  Radiology,  bar-coded  reports  were  prepared, 
verified  and  printed  within  1  day,  with  2-3  days  more  required  for 
report  distribution. 

In  fact,  initially  radiologists  at  NHB  made  greater  use  of  bar 
coding.  Their  motivation  appears  to  have  been  to  reduce  the  backlog 
of  reports  in  transcription  and  produce  reports  in  a  more  timelv 
manner.  In  August  1982,  7  months  after  installation  of  TRIRAD,  92'{  of 
the  24  reports  observed  in  the  Wet  Reading  Room  were  prepared  with  use 
of  bar  coding.  The  interest  in  the  use  of  computer  assistance 
appeared  to  decrease  as  transcription  services  improved  and  the  report 
backlog  decreased.  The  transcriotionists ,  while  excellent  tvpists, 
were  initially  unfamiliar  with  medical  terminology  when  first  hired. 
Over  several  months,  as  they  gained  increasing  experience, 
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radiologists  noted  increased  quality  of  transcribed  reports.  Further, 
a  reduction  in  daily  volume  in  the  summer  of  1982  permitted  the 
transcriptionists  to  reduce  the  back-log  of  incomplete  reports.  The 
resulting  improvement  in  timeliness  for  reports  seems  to  have  been  a 
factor  in  encouraging  radiologists  to  rely  increasingly  on  dictation 
and  transcription,  thus  reducing  the  need  for  computer-assisted 
reporting. 

D.  STAFF  ACCEPTANCE 

Of  49  respondents  to  the  post-implementation  survey  of  Radiology 
Department  staff,  29  staff  (59%  of  all  respondents)  were  satisfied 
overall  with  TRIRAD  while  only  three  staff  (6%)  were  dissatisfied  and 
the  remaining  35%  were  neutral.  The  weighted  mean  response  of  all 
staff  was  +0.7,  indicating  a  satisfaction  level  of  slightly  less  than 
"somewhat  satisfied"  (+1.0). 

A  review  was  made  of  responses  to  particular  survey  questions 
concerning  satisfaction  with  TRIRAD.  A  comparison  was  also  made  of 
baseline  and  pose-implementation  levels  of  satisfaction  with  Radiology 
services  in  general.  These  provided  further  insight  into  the  reasons 
behind  this  overall  level  of  satisfaction. 

Regarding  the  accuracy  of  information,  69%  of  radiologists  were 
dissatisfied  and  only  8%  were  satisfied  with  the  accuracy  of  patient 
records  during  the  baseline  period.  By  comparison,  only  10%  of 
radiologists  were  dissatisfied  and  80%  were  satisfied  with  the 
accuracy  of  patient  records  stored  on  the  computer.  For  all  staff 
surveyed,  the  weighted  mean  response  on  these  questions  improved  from 
-0.3  to  +0.7.  Satisfaction  with  the  completeness  of  patient  records 
improved  from  -0.7  to  +0.4. 

Overall,  staff  agreed  that  information  was  easier  to  obtain  with 
TRIRAD.  On  survey  questions  concerning  the  ease  of  obtaining  reports 
using  TRIRAD,  a  weighted  mean  score  of  +0.8  was  computed  for  the 
post-implementation  survey,  whereas  in  the  baseline  period  the  mean 
staff  responses  were  -0.4  for  ease  of  obtaining  historical  records  and 
-0.3  for  current  records.  In  the  post-implementation  survey,  25 
Radiology  staff  felt  that  attending  physicians'  satisfaction  with 
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Radiology  services  had  increased  as  a  result  of  TRIRAD.  This  is 
believed  to  be  due  in  large  part  to  the  ability  to  ascertain  exam 
status  and  to  obtain  interpretation  reports  from  the  system  instead  of 
searching  for  paper  records.  Interviews  with  attending  physicians 
confirmed  that  they  had  noted  substantial  gains  in  access  to  patient 
Information.  (It  is  important  to  note  that  at  the  time  of  the 
post-implementation  study,  the  historical  data  base  of  patient  exam 
history  and  records  consisted  of  less  than  one  year's  volume;  as  this 
data  base  continues  to  grow,  the  need  to  consult  paper  records 
maintained  prior  to  TRIRAD  should  decrease  accordingly.) 

Overall,  the  Department  staff  surveyed  believed  that  TRIRAD  had 
improved  patient  care;  27  staff  felt  that  there  were  benefits  to 
patient  care  while  12  staff  felt  that  there  was  no  benefit;  residents 
felt  most  strongly  on  this  issue  (eight  felt  there  were  benefits  while 
one  felt  there  was  no  benefit). 

Interviews  with  Radiology  staff  supported  the  survey  findings  on 
improvements  in  patient  care.  Staff  felt  that  the  ability  to  retrieve 
reports  from  the  TRIRAD  system  in  a  more  timely  manner  gave  providers 
the  information  that  they  needed  at  the  time  when  they  needed  it. 
When  the  patient  returned  for  a  follow-up  outpatient  visit,  the 
official  report  was  more  likely  to  be  contained  in  the  patient's 
chart.  Staff  felt  that  the  increased  ability  to  determine  the  status 
of  a  report,  e.g.,  the  report  had  been  entered  by  the  transcriptionist 
but  not  yet  approved*  for  signature,  was  a  benefit.  This  gave  the 
provider  the  information  to  decide  whether  to  come  to  the  Department 
to  discuss  the  film  or  to  wait  for  the  report  to  be  distributed 
through  normal  channels.  Also,  staff  felt  that  the  increased  ability 
to  locate  current  films  also  improved  patient  care.  Often,  an 
attending  physician  will  need  to  view  a  film  whether  he  has  the  report 
or  not.  This  is  common  practice  in  GI  Radiology.  A  surgeon  may  want 
to  borrow  pre-operative  X-rays  for  use  during  the  operative  procedure. 
The  ability  to  locate  these  films  more  easily  and  more  quickly, 
through  the  use  of  the  film-tracking  functions,  was  believed  to  have 
directly  contributed  to  improved  patient  care.  Attending  physicians 
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also  reported  Co  Radiology  staff  that  they  felt  that  the  provider's 
ability  to  care  for  patients  had  disproved  because  of  Improvements  In 
Radiology  services. 

On  several  questions.  Radiology  staff  showed  a  substantial  level 
of  agreement  concerning  the  effect  of  TRIRAD  in  their  work.  Staff 
felt  chat  TRIRAD  made  their  work  more  efficient  (51%  agreed,  38%  were 
neutral,  11%  disagreed).  Record  keeping  was  considered  easier  (55% 
agreed,  26%  were  neutral,  19%  disagreed).  TRIRAD  was  easy  to  learn 
(79%  agreed,  15%  neutral,  6%  disagreed).  Overall,  staff  felt  that  the 
benefits  of  TRIRAD  outweighed  the  disadvantages  (59%  agreed,  24% 
neutral,  17%  disagreed).  Seventy-five  percent  of  Radiology  staff 
believe  that  benefits  from  TRIRAD  would  continue  to  increase  in  the 
future. 
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APPENDIX  A 


EFFECTIVE  RADIOLOGY  SERVICES 
TURNAROUND  TIME 


ROUTINE  REPORTS 
Acceptable  Preferred 


MAIN  RADIOLOGY 


•  STANDARD  EXAMS 


-  Chest 

00 

<r 

Hours 

24 

Hours 

-  Skull 

48 

Hours 

24 

Hours 

-  Spine 

48 

Hours 

24 

Hours 

Extremities 

48 

Hours 

24 

Hours 

•  G.l 

-  UGI 

48 

Hours 

24 

Hours 

-  LGI 

48 

Hours 

24 

Hours 

-  BE 

48 

Hours 

24 

Hours 

IMAGING 

•  CT/HEAD 

-  no  contrast 

48 

Hours 

24 

Hours 

contrast 

48 

Hours 

24 

Hours 

-  contrast  &  no  contrast 

48 

Hours 

24 

Hours 

•  CT/TOTAL  BODY 

-  no  contrast 

48 

Hours 

24 

Hours 

con  -ast 

48 

Hours 

24 

Hours 

contrast  &  no  contrast 

48 

Hours 

24 

Hours 

•  NUCLEAR  MEDICINE 

-  Bone  Scan 

24 

Hours 

8 

Hours 

Cardiac  Renal  Imaging 

48 

Hours 

24 

Hours 

(Computer  Aided) 

-  Lung  Scan 

24 

Hours 

8 

Hours 

Liver,  Brain  etc. 

24 

Hours 

8 

Hours 

-  Thyroid 

48 

Hours 

24 

Hours 

•  ULTRASOUND 

48 

Hours 

24 

Hours 

ORTHOPEDICS 

24 

Hours 

8 

Hours 

SPECIAL  PROCEDURES 


48  Hours  24  Hours 


APPENDIX  B.  TURNAROUND  TIME  FOR  REPORTS 


Tables  B-l  through  B-4  summarize  the  data  obtained  concerning 
turnaround  time  for  reports  in  the  eight  sections  of  the  Radiology  Depart¬ 
ment  at  NHB  in  which  the  TRIRAD  system  was  implemented. 


B-l 


SUMMARY  OF  IIATA  OBTAINED  ON  RFPOHT  TURNAROUND  TIME  FOR  ORTHOPEDIC  KAIIlOMICY  AND  UKOI.UCIC  PAIUOl.OCY 


APPENDIX  C.  INCOMPLETE  REPORT  ANALYSIS 


The  following  pages  present  distribution  graphs  of  the  incomplete 
report  Information  obtained  from  the  TRIRAD  system  during  four  site 
visits,  three  of  which  were  to  monitor  the  status  of  system  implementation 
and  the  last  of  which  coincided  with  the  post-implementation  data  collection. 


VOLUME  OF  COMPLETE  AND  INCOMPLETE  REPORTS 
ALL  RADIOLOGY  SECTIONS 
MONTH  PRECEDING  MARCH  23.  1982 


jo 
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Age  in  Oays  Preceding  March  23,  1962 


VOLUME  OF  COMPLETE  AND  INCOMPLETE  REPORTS 
ORTHOPEDIC  RADIOLOGY  SECTION 
MONTH  PRECEDING  MARCH  23.  1982 


tuodlti  to  MfltOA 


C-4 


Age  in  Oayi  Preceding  March  23, 
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APPENDIX  D.  QUESTIONNAIRE  RESULTS 


On  the  following  pages,  copies  of  the  baseline  and  post-implementa¬ 
tion  questionnaires  are  given.  Tabulated  responses  of  the  Radiology 
Department  staff  who  completed  questionnaires  in  each  survey  have  been 
included;  percentage  of  respondents  giving  each  answer  is  provided,  along 
with  a  weighted  mean  computed  by  use  of  a  weighting  scale  of  +2,  +1,  0, 

-1,  and  -2.  Following  the  post- implementation  questionnaire,  is  a  series 
of  tables  showing  responses  to  questions  regarding  TRIKAD  on  the  post¬ 
implementation  survey.  Responses  are  shown  for  the  Individual  staff 
types  (radiologist,  resident,  technician,  and  other)  and  for  the  individual 
sections  of  the  Radiology  Department. 
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BASELINE  QUESTIONNAIRE 


1.  listed  below  are  general  characteristics  of  radio  logy  da  par  cm  at  oporacloas 
related  to  information  handling.  Please  lodlcata  bow  soelsflod  you  art  with 
•dating  oparaclons  by  racing  aach  leas  Llatad  balow  on  a  sc  a  la  of  "1”  to 
**5"  whara  ’*1”  rapraaanca  vary  dlssaclaflad  and  "S"  is  vary  aaclaflad. 

Vary  Tory  So 

Plaaatlaflad  Satisfied  Keegonse  Haas 


1 

2 

3 

4 

5 

6 

a. 

Tlaaly  availability  of 
paelanca'  currant  records— 
STAT  caaaa 

92 

342 

282 

152 

62 

82 

-0.3 

b. 

Tlaaly  availability  of 
patlants'  currant  records— 
routlna  caaaa 

19Z 

202 

292 

172 

92 

62 

-0.2 

Co 

TlMly  availability  of 
patients'  historical 
records— STAT  cases 

laz 

372 

232 

82 

32 

112 

-0.7 

d. 

Timely  availability  of 
patients'  historical 
records—  routine  cases 

172 

212 

312 

152 

52 

112 

-0.3 

•  o 

Case  of  obtaining  patients' 
currant  records 

17Z 

232 

312 

182 

52 

62 

-0.3 

f. 

Case  of  obtaining  patients' 
historical  records 

17T 

252 

322 

142 

32 

92 

-0.4 

(• 

Timely  availability  of 
patients'  tese  results— 

STAT  cases 

112 

312 

312 

182 

62 

32 

-0.2 

h. 

Timely  availability  of 
patients'  test  results — 
routine  cases 

232 

282 

282  . 

152 

32 

32 

-0.5 

1. 

Ease  of  obtaining  patients' 
test  results— in— patients 

142 

282 

402 

122 

12 

52 

-0.4 

J. 

Case  of  obtaining  patients' 
test  results— telephone 
Inquiries 

252 

352 

212 

112 

32 

52 

-0.7 

k. 

Completeness  of  patients' 
records:  tests  ordered, 
test  results 

232 

262 

352 

82 

22 

6* 

-0.7 

1. 

Accuracy  of  patients" 
records 

122 

212 

482 

92 

52 

57. 

-0.3 

So 

Leglbllicy/ease  of 
reading  patience' 
records 

282 

282 

262 

112 

42 

32 

-0.7 

A. 

Amount  of  paperwork/ 
time  required  for 
writing  patients’ 
orders 

142 

192 

322 

182 

32 

142 

-0.2 

0# 

Completeness  of 
departnenc  files, 
l.e., number  of 
lost  files 

312 

372 

172 

82 

32 

42 

-0.9 

p- 

Tima  available  for 
patient  cars 

62 

262 

312 

282 

12 

82 

-0.1 
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The  following  statements  describe  how  an  Individual  may  react  to  a 
computerized  radiology  management  system.  Please  Indicate  to  what 
extent  each  statement  describes  how  you  personally  feel.  The  more 
you  agree  with  a  statement,  the  higher  the  number  you  should  give  It. 


Strongly 
01 sagree 

Strongly 
Agree  . 

Mo 

Response 

Mean 

a. 

It  will  be  time-consuming 
for  me  to  learn 

T 

152 

2 

22Z 

3 

352 

4 

17Z 

5 

82 

32 

-0.2 

b. 

It  will  be  difficult  for  me 
to  use  the  terminal 

4QZ 

34Z 

17Z 

32 

32 

32 

-1.1 

c. 

It  will  make  my  work  more 
efficient 

s: 

6Z 

352 

222 

292 

32 

0.7 

d. 

I  will  not  trust  the 
Information  I  obtain 

46: 

31Z 

172 

32 

02 

32 

-1.2 

e. 

It  will  make  record  keeping 
more  difficult  for  me 

32Z 

39Z 

232 

32 

02 

32 

-1.0 

f. 

I  personally  will  use  It  as 
little  as  possible 

46  Z 

232 

182 

52 

52 

32 

-1.0 

g. 

It  will  make  my  work  more 
challenging 

112 

24Z 

342 

112 

152 

52 

0.0 

h. 

The  whole  Idea  of  a  radiology 
management  system  turns  me  off 

66Z 

142 

112 

32 

'lZ 

52 

-1.5 

i. 

The  benefits  to  me  will  outweigh 
the  trouble  of  learning  to  use 
the  system  3Z 

52 

222 

292 

00 

CO 

32 

1.0 

j. 

It  will  significantly  alter 
the  way  I  do  my  work 

9% 

12Z 

372 

222 

152 

52 

0.2 

<. 

It  will  make  writing  orders 
for  my  patients  easier/less 
time-consuming 

6" 

92 

352 

172 

52 

23Z 

0.1 

i. 

It  will  be  difficult  for  me 
to  read  the  CRT  terminal  screen  512 

212 

142 

32 

32 

82 

-1.2 

m. 

I  will  be  able  to  provide 
better  service  to  my  patients 

2Z 

62 

252 

402 

212 

62 

0.8 

n. 

I  will  have  less  flexibility 
because  operations 
will  be  more  routinlzed 

28Z 

252 

322 

92 

12 

52 

-0.7 

0. 

Some  decision-making  will 
oc  taken  away  from  me 

297. 

312 

21 

92 

52 

52 

-0.7 

P. 

I  will  have  more  time  for 
direct  patient  care 

62 

142 

402 

202 

62 

142 

0.1 
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3-  As  you  may  know,  a  radiology  management  system  can  Incorporate  a  number 
of  dlfforont  functions.  For  tach  of  the  functions  listed  below,  please 
indicate  how  Important  or  desirable  It  Is  to  you  to  have  that  function 
or  capability  In  the  radiology  management  system  In  this  department. 
Rate  each  function  on  a  scale  of  *1"  to  "5",  where  “1“  represents 
not  at  all  Important  and  “5"  Is  very  important. 


• 

Not  At  All 
Important 

Very 

Important 

No 

Response 

Mean 

1 

2 

3 

a 

5 

a. 

Scheduling  of  In-patient  tests 

sz 

6Z 

11Z 

23% 

55Z 

OZ 

1.2 

b. 

Scheduling  of  out-patient  tests 

SZ 

SZ 

I5Z 

26Z 

48Z 

1Z 

1.1 

c. 

Patient  registration 

3Z 

5Z 

17Z 

21Z 

51Z 

:z 

1.2 

- 

d. 

Record/track  to  whom/where 
patient  x-ray  file  folders 
signed  out 

1Z 

OZ 

SZ 

3Z 

91Z 

oz 

1.8 

e. 

Place  test  orders 

3Z 

14Z 

27Z 

17Z 

31Z 

8% 

0.6 

f. 

Record  test  results 

oz 

11Z 

14Z 

20Z 

52Z 

3Z 

1.2 

9- 

Correlate  test  results  with 
previous  tests/other  data 

6% 

17T 

23Z 

49Z 

3% 

1.2 

h. 

Record  radiologist' s/other 
physician's  evaluations 

3* 

3Z 

9Z 

23Z 

60Z 

2Z 

1.4 

• 

1. 

Maintain  patient  records  on 
current  hospital  stay  (tests 
ordered,  test  results/evalua¬ 
tions) 

2Z 

3Z 

I5Z 

35Z 

43Z 

2% 

1.2 

J. 

Maintain  ongoing  data 
file  on  each  patient  — 
registration  data 

2 Z 

3Z 

17Z 

32Z 

43Z 

3% 

1.2 

* 

k. 

Maintain  ongoing  data  file 
on  each  patient— medical  data 
(historical  record  of  visits, 
test  results,  treatments,  etc.) 

1% 

SZ 

21Z 

31Z 

40Z 

■>» 

1.0 

* 

1. 

Maintain/generate  radiology 
department  statistics  (number 
of  patients,  number  of  tests, 

OZ 

3Z 

20Z 

26Z 

43% 

3% 

1.2 

time  per  test,  etc.) 


4.  Listed  below  are  possible  benefits  a  computerized  radiology  management 
system  might  have.  For  each  of  these,  please  Indicate  to  what  extent  you 
believe  that  benefit  will  be  seen  In  the  radiology  department  operations 
of  this  hospital  when  the  radiology  management  system  Is  Installed.  The 
more  likely  you  think  It  Is  that  benefit  will  come  about,  the  higher  the 
number  you  should  give  It. 


*  Definitely 

A  computerized  radiology  Will  Not 

Definitely 

Will 

So 

Response 

Mean 

management  system  will... 

1 

_2 

3 

4 

5 

a. 

Reduce  the  time  required  to 
produce  a  written  report 

32 

92 

162 

342 

352 

32 

0.9 

b. 

Reduce  the  number  of  lost  files 

22 

92 

172 

312 

412 

02 

1.0 

c. 

Result  In  more  complete 
patient  records 

02 

52 

172 

412 

372 

02 

1.1 

"  d. 

Make  patients*  current 
records  more  easily  accessible 

22 

32 

122 

452 

382 

02 

1.2 

e. 

Make  patients'  historical 
records  more  easily  accessible 

02 

62 

232 

392 

322 

02 

1.0 

f. 

Make  patients'  current  records 
available  more  quickly 

02 

52 

182 

432 

‘  342 

02 

1.1 

S- 

*ake  patients'  historical 
records  available  more  quickly 

•  02 

82 

222 

00 

m 

322 

02 

1.0 

n. 

Reduce  the  number  of  errors 
in  patient  records 

32 

112 

382 

212 

252 

22 

0.5 

1. 

Make  radiology  function 
more  smoothly 

12 

62 

342 

342 

252 

or 

0.7 

* 

*  • 

Allow  better  patient  care 

52 

112 

322 

282 

242 

02 

0.6 

Sc. 

Oecrease  the  time  spent  looking 
for  patient  records/test  results 

02 

r\+ 

142 

382 

482 

02 

1.3 

i. 

Improve  scheduling  of  patients 

02 

32 

232 

392 

352 

02 

1.1 

X 

m. 

Oecrease  the  cost  of  radiology 
services 

92 

142 

382 

252 

112 

32 

0.1 

n. 

Improve  the  way  things 
are  done  here 

52 

62 

292 

312 

292 

02 

0.7 

♦ 

0. 

Reduce  the  number  of  people 
needed  to  work  here 

342 

352 

292 

112 

12 

C2 

-0.8 

P. 

Make  reports  easier  to  read 

02 

232 

372 

382 

02 

1.1 

d. 

Reduce  the  numoer  of  telephone 
Inquiries  for  test  results  due 
to  Improved  turnaround 

12 

92 

252 

282 

372 

02 

0.9 
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5.  How  familiar  are  you  with  TUI 3 AO  or  other  radiology  management  systems? 
(Please  circle  all  that  apply.) 


122  2 

62  3 
lit  4 

9t  S 

a:  6 
3*2  7 

IK  8 
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Have  personally  used  or  tried  TRIRAO  In  other  hospital 

Have  personally  used  or  tried  other  radiology  management  system 

In  other  hospital 

Have  seen  TRIRAO  in  use  In  other  hospital,  but  have  not  personally  used 
Have  seen  other  radiology  management  system  In  use  In  other  hospital, 
but  have  not  personally  used 
Have  seen  TP I RAO  demonstrated 

Have  seen  other  radiology  management  system  demonstrated 

Have  read  or  heard  about  TRIRAO  but  have  not  seen  demonstrated  or  used 

Have  read  or  heard  about  other  radiology  management  system  but  have 

not  seen  demonstrated  or  used 

No  familiarity  or  experience  with  TRIRAO 

No  familiarity  or  experience  with  other  radiology  management  system 
Mo  Response 


Finally,  for  background  purposes,  please  specify  the  following: 


6.  Your  specialty: 

2021  Staff  radiologist 
622  Radiology  resident  -  1st  year 
523  Radiology  resident  -  2nd  year 
12*  Radiology  resident  -  3rd  year 
5725  Radiology  technologist  • 

1126  Other 

027  Mo  Response  a 

7.  Your  section  In  this  hospital: 

4921  Diagnostic  Radiology 
1222  Imaglng/CT 
2023  Imaging/Nuclear  Medicine 
1124  Imaging/Ultrasound 
1825  Radiation  Therapy 

526  Special  Procedures 

527  Orthopedl c$ 

028  Urologyb 

229  Refused/Mo  Response 


a.  The  percentage  of  staffing  in  each  section  total  to  more  than  100% 
since  some  staff  work  in  more  than  one  section  and  thus  selected 
more  than  one  response. 

b.  Questionnaires  were  not  distributed  to  Urology  staff. 
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POST-IMPLEMENTATION  QUESTIONNAIRE 


Listed  below  are  general  characteristics  of  radiology  department  operations 
related  to  information  handling.  Please  Indicated  how  satisfied  you  are  now 
with  current  operations  by  rating  each  item  listed  below  on  a  scale  of  "1”  to 
"5"  where  "1"  represents  very  dissatisfied  and  "5"  is  very  satisfied. 

Very  Very  No 

Dissatisfied  Satisfied  Response  Mean 


1 

2 

3 

4 

5 

Timely  availability  of  patients' 
current  films 

6% 

17% 

36% 

34% 

7% 

0% 

.21 

Timely  availability  of  patients' 
current  reports 

4 

15 

32 

28 

21 

0 

.47 

Timely  availability  of  patients' 
historical  films 

17 

15 

41 

23 

4 

0 

-.19 

Timely  availability  of  patients' 
historical  reports 

11 

15 

25 

28 

17 

4 

.25 

Ease  of  obtaining  films 

6 

22 

34 

36 

2 

0 

.06 

Ease  of  obtaining  reports 

4 

9 

19 

42 

26 

0 

.77 

Completeness  of  patients'  exam 
history:  tests  ordered,  test 
results 

10 

10 

28 

37 

15 

0 

.40 

Accuracy  of  patients'  records 
stored  on  computer 

6 

4 

22 

51 

17 

0 

.70 

Ease  of  getting  access  to  terminal 

9 

21 

15 

32 

19 

4 

.31 

Completeness  of  department  files, 
i.e.,  number  of  lost  files 

11 

23 

36 

26 

2 

2 

-.15 

Now  that  the  TRIRAD  radiology  management  system  has  been  in  place  for  nine 
months,  how  would  you  describe  your  overall  satisfaction  with  the  system? 


5 

Very  satisfied 

11% 

4 

Fairly  satisfied 

43 

3 

Mixed 

32 

2 

Fairly  dissatisfied 

6 

1 

Very  dissatisfied 

0 

No  response 

8 
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[  3.  Now  chat  you  have  had  the  opportunity  to  work  with  the  TRIRAD  system,  you 
are  in  a  position  to  judge  the  benefits  a  computerized  radiology  management 
system  might  or  might  not  have  over  a  manual  system.  For  each  of  these, 
please  indicated  to  what  extent  you  believe  that  benefit  has  been  seen  in  the 
I  radiology  department  operations  of  this  hospital  now* that  the  TRIRAD  system 
is  operational.  The  more  convinced  you  are  chat  each  benefit  has  come  about, 
the  higher  the  number  you  should  give  It. 


Definitely 

The  TRIRAD  computerized  Has  Not 

Definitely 

Has 

No 

Response 

Mean 

radiology  management  system 

has  .  .  .  1 

2 

3 

4 

5 

a.  Reduced  the  time  required  to 

produce  a  written  report  9% 

11% 

9% 

27% 

42% 

2% 

.81 

b.  Reduced  the  number  of  lost  files  13 

19 

36 

26 

4 

2 

-.12 

c. 

« 

% 

Resulted  in  more  complete 
patient  records 

0 

8 

24 

43 

21 

4 

.  80 

d. 

Made  patients'  historical 
records  more  easily  accessible 

2 

6 

24 

34 

34 

0 

.92 

-  e. 

Made  patient'  historical 
records  available  more  quickly 

0 

9 

21 

38 

32 

0 

.92 

f. 

Reduced  the  number  of  errors 
in  patient  records 

9 

11 

47 

25 

6 

2 

.  06 

8* 

Made  radiology  function 
more  smoothly 

11 

6 

38 

34 

9 

2 

.  25 

h. 

Improved  patient  care 

11 

11 

26 

44 

8 

0 

•  25 

i. 

Decreased  the  time  spent  looking 
for  patient  records/test  results 

4 

9 

4 

53 

28 

2 

•  94 

*  j- 

Improved  scheduling  of  patients 

8 

4 

22 

38 

22 

6 

.  68 

k. 

Decreased  the  cost  of  radiology 
services 

11 

13 

46 

13 

2 

15 

-  .  22 

*  1. 

Reduced  the  number  of  people 
needed  to  work  here 

43 

25 

15 

11 

4 

2 

-  .  94 

m. 

Made  reports  easier  to  read 

0 

2 

13 

26 

59 

0 

1  .  42 

n. 

Rr iuced  the  number  of  telephone 
inquiries  for  test  results  due 
to  Improved  turnaround 

13 

13 

28 

34 

8 

4 

.  10 

0. 

Reduced  the  number  of  unneces¬ 
sary  repeat  examinations 

4 

21 

34 

28 

9 

4 

.  20 

P- 

Increased  satisfaction  level  of 
attending  physicians  with 
radiology  service 

4 

6 

36 

36 

11 

7 

.  49 

q* 

Increased  the  accuracy  of 
radiology  reports 

8 

11 

28 

30 

17 

6 

•  40 
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4.  Please  indicate  to  what  extent  each  statement  describes  how  you  personally  wer? 
affected  by  the  computerized  radiology  management  system.  The  more  you  agree 


with  a  statement,  the  higher  the  number  you  should  give 

Strongly 

Disagree 

it. 

Strongly 

Agree 

No 

Response 

Mean 

1 

2 

3 

4 

5 

.  a. 

It  was  easy  for  me  to  learn  to 
use  the  radiology  system 

2% 

4% 

15% 

24% 

55% 

0% 

1.26 

b. 

It  has  made  my  work  more  efficient 

2 

9 

38 

30 

21 

0 

.58 

c. 

I  do  not  trust  the  information  1 
obtain 

44 

28 

13 

9 

4 

2 

1.00 

d. 

It  has  made  record  keeping  easier 
for  me 

9 

9 

27 

51 

4 

0 

.30 

e. 

I  personally  try  to  use  it  as 
little  as  possible 

58 

25 

11 

2 

4 

0 

1.32 

f. 

It  has  made  my  work  more  challenging 

15 

45 

10 

15 

0  -  .06 

g* 

The  benefits  to  me  have  outweighed 
the  disadvantages  of  the  system 

11 

6 

22 

38 

19 

4 

.49 

h. 

It  has  significantly  altered  the 
way  I  do  my  work 

8 

21 

28 

26 

15 

2 

.21 

i. 

I  have  been  able  to  provide  better 
service  to  my  patients 

8 

9 

44 

26 

11 

2 

.25 

*  j. 

I  have  had  less  flexibility  because 
operations  are  more  routinized 

15 

28 

36 

9 

8 

4 

.35 

k. 

4 

Some  decision-making  has  been  taken 
away  from  me 

22 

30 

32 

6 

6 

4 

•  61 

1, 

The  system  saves  me  time 

15 

13 

26 

25 

19 

2 

.  19 

m. 

i 

I  expect  that  benefits  from  this 
system  will  increase  in  the  future 

6 

2 

17 

36 

39 

0 

1.02 

5.  Indicate  which  functions  of  the  computer  you  use  most  often. 
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6.  Add  any  further  comments,  positive  or  negative,  about  the  system  Itself  or  the 
way  in  which  it  is  used.  Please  elaborate  further  or  address  questions  which 
we  did  not  ask. 


Finally,  for  background  purposes,  please  specify  the  following: 
7  .  Your  specialty: 


1. 

Staff  radiologist 

19% 

2. 

Radiology  resident  -  1st  year 

11 

3. 

Radiology  resident  -  2nd  year 

4 

4. 

Radiology  resident  -  3rd  year 

4 

5. 

Radiology  technologist 

41 

6. 

Other  (please  specify) : 

21 

8.  Your  section  in  this  hospital: 

1. 

2. 

.  3. 

4. 

5. 

6. 

7. 

* 

9.  Were  you  employed  in  this  hospital  before  January  1982  when  the  TRIRAD  system 
was  installed? 

'  1.  Yes  74% 

2.  No  26 

No  Response  0 


*Note  -  Staff  in  imaging  tend  to  rotate  through  the  various  imaging  sections, 

therefore  the  individual  percentiles  reflect  their  interaction  with  each 
section;  the  45%  indicates  their  representation  in  the  total  53  respondents. 


Diagnostic  Radiology 
Imaging/CT 

Imaging/Nuclear  Medicine 
Imaging /Ultrasound 
Special  Procedures 
Orthopedics 
Orology 


42% 

15  ) 
30  } 

9  ) 
6 

4  ' 
4  < 


45%* 


13% 
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If  you  were  employed  here  before  the  TRIRAD  system  was  Installed,  please 
answer: 

10.  Has  your  job  function  or  title  changed  since  the  TRIRAD  system  was 
installed? 


1.  Yes  36X 

2.  No  64 

No  Response  0 

If  your  lob  function  or  title  has  changed,  please  answer  Qs.  11  and  12: 

11.  Please  explain  what  your  function  and  title  was  before. 


12.  Uas  the  change  in  your  job  function  or  title  directly  related 
to  the  implementation  of  the  computerized  management  system? 

1.  Yes  7% 

2.  No  86 

No  Response  7 


13.  Your  name: 


Thank  you  for  your  time  and  effort. 
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APPENDIX  E.  COST  INFORMATION 


Table  E-l  presents  annual  costs  for  hardware,  software,  and 
maintenance  of  TRIRAD  at  NHB,  from  FY82  through  FY89.  One-time  costs 
applied  In  FY82  are  summarized  In  Table  E-2. 
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TABLE  E-2 


TRIRAD  SYSTEM 
ONE-TIME  CHARGES  FOR  FY82 
NAVAL  HOSPITAL,  BETHESDA 


One-Time  Charges 

Documentation: 

Computer  Operator  Manual 
Functional  User  1  Manual 
Functional  User  2  Manual 
System  Description  Manual 
Terminal  Operator  Manual 
Training  Materials 

Total  Documentation  Costs 

Software  Items  License  Fees: 

Software  Film  File  Management 
Software  Patient  Record  File 
Raport  IIA 
Raport  IIB 
Scheduling  Feature 
Statistical  Package 
System  Initialization 
Teaching  File 

Total  Software  Items  License  Fees 

Training  Costs: 

Computer  Operator  Class 
Executive  Orientation  Class 
File  and  Table  Building  Class 
Functional  User  Class2 
Functional  User  Class 

Total  Training  Costs 

Total  One-Tine  Charges 


Cost /Item 
or 


Monthly  Cost 

Total 

Number 

Per  Item 

Cost 

3 

$  15.00 

$  45.00 

13 

15.00 

195.00 

13 

15.00 

195.00 

3 

10.00 

30.00 

13 

10.00 

130.00 

135 

5.00 

675.00 

S  1,270.00 

- 

- 

$  11,250.00 

- 

- 

15,000.00 

- 

• 

15,000.00 

- 

- 

11,250.00 

- 

- 

11,250.00 

- 

- 

7,500.00 

- 

- 

7,000.00 

- 

7,500.00 

$  85,750.00 

1 

S  1,390.00 

$  1,390.00 

1 

1,390.00 

1,390.00 

2 

1,390.00 

2,780.00 

10 

1,390.00 

13,900.00 

3 

1,390.00 

4,170.00 

$  23,630.00 

$  110,650.00 

Conducted  after  system  installation 
Conducted  after  optical  scanner  installation 
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APPENDIX  F.  DAILY  DEPARTMENT  WORKLOAD 
DURING  POST-IMPLEMENTATION  PERIOD 

Table  F-l  provides  the  daily  volume  of  completed  procedures  in 
each  of  the  eight  sections  of  the  Department  in  which  TRIRAD  was  imple¬ 
mented  for  the  period  of  post- implementation  data  collection. 


I 

I 


F-l 


